\[ y'(x)-y(x)^2-y(x) \sin (2 x)-\cos (2 x)=0 \] ✓ Mathematica : cpu = 0.508802 (sec), leaf count = 113
\[\left \{\left \{y(x)\to -\frac {-\sin (x) \int _1^{\cos (x)}\frac {e^{-K[1]^2}}{K[1]^2 \sqrt {K[1]^2-1}}dK[1]-\frac {e^{-\cos ^2(x)} \tan (x)}{\sqrt {\cos ^2(x)-1}}-c_1 \sin (x)}{\cos (x) \int _1^{\cos (x)}\frac {e^{-K[1]^2}}{K[1]^2 \sqrt {K[1]^2-1}}dK[1]+c_1 \cos (x)}\right \}\right \}\] ✓ Maple : cpu = 0.748 (sec), leaf count = 128
\[y \relax (x ) = \frac {2 \sin \left (2 x \right ) \left (c_{1} \left (\cos \left (2 x \right )+1\right ) \HeunCPrime \left (1, \frac {1}{2}, -\frac {1}{2}, -1, \frac {7}{8}, \frac {\cos \left (2 x \right )}{2}+\frac {1}{2}\right )+\HeunC \left (1, \frac {1}{2}, -\frac {1}{2}, -1, \frac {7}{8}, \frac {\cos \left (2 x \right )}{2}+\frac {1}{2}\right ) c_{1}+\frac {\HeunCPrime \left (1, -\frac {1}{2}, -\frac {1}{2}, -1, \frac {7}{8}, \frac {\cos \left (2 x \right )}{2}+\frac {1}{2}\right ) \sqrt {2 \cos \left (2 x \right )+2}}{2}\right )}{\sqrt {2 \cos \left (2 x \right )+2}\, \left (c_{1} \HeunC \left (1, \frac {1}{2}, -\frac {1}{2}, -1, \frac {7}{8}, \frac {\cos \left (2 x \right )}{2}+\frac {1}{2}\right ) \sqrt {2 \cos \left (2 x \right )+2}+\HeunC \left (1, -\frac {1}{2}, -\frac {1}{2}, -1, \frac {7}{8}, \frac {\cos \left (2 x \right )}{2}+\frac {1}{2}\right )\right )}\]
Hand solution
\begin {align} y^{\prime }-y^{2}-y\sin \left (2x\right ) -\cos \left (2x\right ) & =0\nonumber \\ y^{\prime } & =y^{2}+y\sin \left (2x\right ) +\cos \left (2x\right ) \tag {1} \end {align}
This is Riccati first order non-linear ODE of the form of the general form \(y^{\prime }=P\relax (x) +Q\relax (x) y+R\relax (x) y^{2}\) where \(P\relax (x) =\cos \left (2x\right ) ,Q\relax (x) =\sin \left (2x\right ) ,R\relax (x) =1\). It is best to first try to spot a particular solution \(y_{p}\) and use the transformation \(y=y_{p}+\frac {1}{u}\) otherwise we use \(y=-\frac {u^{\prime }}{yR\relax (x) }\) transformation. For this problem \[ y_{p}=\tan \relax (x) \]
To verify, since \(y_{p}^{\prime }=\frac {1}{\cos ^{2}x}\) then plugging this particular in (1) gives
\[ \frac {1}{\cos ^{2}x}-\tan ^{2}\relax (x) -\tan \relax (x) \sin \left (2x\right ) -\cos \left (2x\right ) =0 \]
But \(\cos \left (2x\right ) =\cos ^{2}x-\sin ^{2}x\) and \(\sin \left (2x\right ) =2\sin x\cos x\) and \(\tan \relax (x) =\frac {\sin x}{\cos x}\) therefore the above becomes
\begin {align*} \frac {1}{\cos ^{2}x}-\frac {\sin ^{2}x}{\cos ^{2}x}-\frac {\sin x}{\cos x}\left ( 2\sin x\cos x\right ) -\left (\cos ^{2}x-\sin ^{2}x\right ) & =0\\ \frac {1}{\cos ^{2}x}-\frac {\sin ^{2}x}{\cos ^{2}x}-2\sin ^{2}x-\cos ^{2}x+\sin ^{2}x & =0\\ \frac {1}{\cos ^{2}x}-\frac {\sin ^{2}x}{\cos ^{2}x}-\sin ^{2}x-\cos ^{2}x & =0\\ \frac {1}{\cos ^{2}x}-\frac {\sin ^{2}x}{\cos ^{2}x}-1 & =0\\ \frac {1-\sin ^{2}x}{\cos ^{2}x}-1 & =0\\ \frac {\cos ^{2}x}{\cos ^{2}x}-1 & =0\\ 1-1 & =0\\ 0 & =0 \end {align*}
Therefore we, we can use \(y=y_{p}+\frac {1}{u}\)
\begin {align*} y & =\tan x+\frac {1}{u}\\ y^{\prime } & =\frac {1}{\cos ^{2}x}-\frac {u^{\prime }}{u^{2}} \end {align*}
Equating this to (1) gives
\begin {align*} -\frac {u^{\prime }}{u^{2}} & =y^{2}+y\sin \left (2x\right ) +\cos \left ( 2x\right ) \\ -\frac {u^{\prime }}{u^{2}} & =-\frac {1}{\cos ^{2}x}+\left (\tan x+\frac {1}{u}\right ) ^{2}+\left (\tan x+\frac {1}{u}\right ) \sin \left (2x\right ) +\cos \left (2x\right ) \end {align*}
Using \(\sin \left (2x\right ) =2\sin x\cos x\) and \(\cos 2x=\cos ^{2}x-\sin ^{2}x\) then above becomes
\begin {align*} -\frac {u^{\prime }}{u^{2}} & =-\frac {1}{\cos ^{2}x}+\left (\tan ^{2}x+\frac {1}{u^{2}}+\frac {2}{u}\tan x\right ) +\left (\frac {\sin x}{\cos x}+\frac {1}{u}\right ) 2\sin x\cos x+\left (\cos ^{2}x-\sin ^{2}x\right ) \\ u^{\prime } & =\frac {u^{2}}{\cos ^{2}x}-\left (u^{2}\frac {\sin ^{2}x}{\cos ^{2}x}+1+2u\frac {\sin x}{\cos x}\right ) -\left (u^{2}\frac {\sin x}{\cos x}+u\right ) 2\sin x\cos x-u^{2}\cos ^{2}x+u^{2}\sin ^{2}x\\ & =\frac {u^{2}}{\cos ^{2}x}-u^{2}\frac {\sin ^{2}x}{\cos ^{2}x}-1-2u\frac {\sin x}{\cos x}-2u^{2}\frac {\sin x}{\cos x}\sin x\cos x-2u\sin x\cos x-u^{2}\cos ^{2}x+u^{2}\sin ^{2}x\\ & =\frac {u^{2}}{\cos ^{2}x}-u^{2}\frac {\sin ^{2}x}{\cos ^{2}x}-1-2u\frac {\sin x}{\cos x}-2u^{2}\sin ^{2}x-2u\sin x\cos x-u^{2}\cos ^{2}x+u^{2}\sin ^{2}x\\ & =\frac {u^{2}}{\cos ^{2}x}-u^{2}\frac {\sin ^{2}x}{\cos ^{2}x}-1-2u\frac {\sin x}{\cos x}-u^{2}\sin ^{2}x-2u\sin x\cos x-u^{2}\cos ^{2}x\\ & =u^{2}\left (\frac {1}{\cos ^{2}x}-\frac {\sin ^{2}x}{\cos ^{2}x}-\left ( \sin ^{2}x+\cos ^{2}x\right ) \right ) -1+u\left (-2\frac {\sin x}{\cos x}-2\sin x\cos x\right ) \\ & =u^{2}\left (\frac {1-\sin ^{2}x}{\cos ^{2}x}-1\right ) -1+u\left ( -2\frac {\sin x}{\cos x}-2\sin x\cos x\right ) \\ & =u^{2}\left (\frac {\cos ^{2}x}{\cos ^{2}x}-1\right ) -1+u\left ( -2\frac {\sin x}{\cos x}-2\sin x\cos x\right ) \\ & =-1+2u\left (-\frac {\sin x}{\cos x}-\sin x\cos x\right ) \end {align*}
Hence
\[ u^{\prime }+2u\left (\tan x+\sin x\cos x\right ) =-1 \]
Integrating factor is \(e^{2\int \tan x+\sin x\cos xdx}\). But \[ \int \tan xdx=-\ln \left (\cos x\right ) \] And \[ \int \sin x\cos xdx=\frac {-1}{2}\cos ^{2}x \] Hence \(\mu =e^{-2\ln \cos x}e^{-\cos ^{2}x}=\frac {1}{\cos ^{2}x}e^{-\cos ^{2}x}\), therefore
\[ d\left (\frac {1}{\cos ^{2}x}e^{-\cos ^{2}x}u\right ) =\frac {-1}{\cos ^{2}x}e^{-\cos ^{2}x}\]
Integrating both sides
\begin {align*} \frac {1}{\cos ^{2}x}e^{-\cos ^{2}x}u & =-{\displaystyle \int } \frac {e^{-\cos ^{2}x}}{\cos ^{2}x}dx+C\\ u & =\cos ^{2}xe^{\cos ^{2}x}\left (C-{\displaystyle \int } \frac {e^{-\cos ^{2}x}}{\cos ^{2}x}dx\right ) \end {align*}
Since \(y=\tan x+\frac {1}{u}\) then
\begin {align*} y & =\tan x+\frac {1}{\cos ^{2}xe^{\cos ^{2}x}\left (C-{\displaystyle \int } \frac {e^{-\cos ^{2}x}}{\cos ^{2}x}dx\right ) }\\ & =\tan x+\frac {e^{-\cos ^{2}x}}{\cos ^{2}x}\left (C-{\displaystyle \int } \frac {e^{-\cos ^{2}x}}{\cos ^{2}x}dx\right ) ^{-1} \end {align*}
I do not know how Maple came up with the solution involving HeunC functions since \({\displaystyle \int } \frac {e^{-\cos ^{2}x}}{\cos ^{2}x}dx\) has no closed form solution. I should ask CAS experts about this.
Below is screen shot from Kamke book of the solution it gives, which matches the above result