You have:
$$\int \cos{\theta} \cdot (\sin(3\theta)+\sin{\theta})\cdot e^{\sin{\theta}} ~d\theta$$
Note that $\sin(3\theta)=3\sin{\theta}\cos^2{\theta}-\sin^3{\theta}$.
Due to the identity:
$$\cos^2{\theta}+\sin^2{\theta} \equiv 1$$
We can obtain an expression for $\sin{3\theta}$ in terms of $\sin{\theta}$ by rearranging the above identity.
$$\sin(3\theta)=3\sin{\theta}(1-\sin^2{\theta})-\sin^3{\theta}=3\sin\theta-3\sin^3{\theta}-\sin^3{\theta}=3\sin{\theta}-4\sin^3{\theta}$$
Hence, we will now substitute this into our integral.
$$\int \cos{\theta} \cdot (4\sin{\theta}-4\sin^3{\theta})\cdot e^{\sin{\theta}} ~d\theta=4 \int \cos{\theta} \cdot (\sin{\theta}-\sin^3{\theta})\cdot e^{\sin{\theta}} ~d\theta \tag{1}$$
As your book has stated, substituting $\sin{\theta}=t$ is useful. To do so, we must also obtain an expression with $d\theta$ in order to change the variable the integration it is respect to.
To do so is simple, differentiate your substitution with respect to $\theta$.
$$\color{blue}{t=\sin{\theta}}$$
$$\frac{dt}{d\theta}=\cos{\theta}$$
$$\color{red}{dt=\cos{\theta}~d\theta}$$
Now we can substitute on equation $(1)$!
$$4 \int \color{red}{\cos{\theta}} \cdot \color{blue}{(\sin{\theta}-\sin^3{\theta})\cdot e^{\sin{\theta}}} ~\color{red}{d\theta} =4 \int (t-t^3)e^t ~dt$$
We integrate by parts 3 times to obtain:
$$4 \int (t-t^3)e^t ~dt=-4e^t(t^3-3t^2+5t-5)+C$$
We now substitute back for $t=\sin{\theta}$.
$$-4e^t(t^3-3t^2+5t-5)+C=-4e^{\sin{\theta}}(\sin^3{\theta}-3\sin^2{\theta}+5\sin{\theta}-5)+C$$
We can rearrange this solution into your form to find the coefficients $A,B,C,D,E$.
Reminder:
$$\sin^2{\theta}+\cos^2{\theta} \equiv 1$$
Hence, we can replace the $\sin^2{\theta}$ in terms of $\cos^2{\theta}$ since that is the form of your equation of undetermined coefficients.
$$=-4e^{\sin{\theta}}(\sin^3{\theta}-3(1-\cos^2{\theta})+5\sin{\theta}-5)+C$$
$$=-4e^{\sin{\theta}}(\sin^3{\theta}+3\cos^2{\theta}+5\sin{\theta}-8)+C$$
$$=e^{\sin{\theta}}(-4\sin^3{\theta}-12\cos^2{\theta}-20\sin{\theta}+32)+C$$
Hence, you have your solution for the coefficients:
$A=-4$, $B=-12$, $C=-20$, $D=0$, $E=32$ and obviously $F=C$.
If you have any further questions or doubts, do not hesitate to ask.