$$y''+y=cos3x$$
$$\lambda^2+\lambda=0\Rightarrow \lambda_{1,2}=\pm i$$
$$y_{h}=c_{1}cosx+c_{2}sinx$$
Because the homogeneous solutions is $ c_{1}cosx $ and not $c_{1}cos3x$ we can the particular solution to be:
$$y_{p}=Acos3x+Bsin3x$$
Substituting the particular solution to the ODE:
$$y'_{p}=-3Asin3x+3Bcos3x$$
$$y''_{p}=-9Acos3x-9Bsin3x$$
$$-9Acos3x-9Bsin3x+Acos3x+Bsin3x=cos3x$$
$$-8Acos3x-8Bsin3x=cos3x$$
Is it correct that there is no sin3x on the RHS of the equation? usually it is because the particular solution was incorrect