hint:
For part (a)
$$2(\cos (\alpha) +\sin(\alpha))\cos \theta + 4(\cos \alpha - \sin \alpha)\sin(\theta) = k$$
By harmonic addition theorem, http://mathworld.wolfram.com/HarmonicAdditionTheorem.html
I can express the $a \cos \theta + b \sin \theta$ in the form of $R cos(\theta + \delta)$ where $R^2=a^2+b^2$.
\begin{align}R^2 &= 4(\cos (\alpha) + \sin(\alpha))^2+16(\cos \alpha - \sin \alpha)^2\\&=4+4\sin(2\alpha)+16-16\sin(2\alpha)\\
&=20-12\sin(2\alpha)
\leq 32\end{align}
Hence $k^2 \leq 32$.
For part (b)
$$2(\cos (\alpha) +\sin(\alpha))\cos \theta + 4(\cos \alpha - \sin \alpha)\sin(\theta) = 2\sqrt{2}$$
$$sign(2(\cos(\alpha)+\sin(\alpha))\sqrt{20-12\sin(2\alpha)}\cos\left(\theta+\tan^{-1}\left(-\frac{4(\cos\alpha-\sin\alpha)}{2(\cos\alpha+\sin\alpha)}\right)\right)=2\sqrt{2}$$
$$\cos\left(\theta+\tan^{-1}\left(-\frac{4(\cos\alpha-\sin\alpha)}{2(\cos\alpha+\sin\alpha)}\right)\right)=\frac{2\sqrt{2}}{sign(2(\cos(\alpha)+\sin(\alpha))\sqrt{20-12\sin(2\alpha)}}$$
Homework: check that the denominator is non-zero. If it is zero, handle that case separately.
Hence the system is solvable if
$$\left| \frac{2\sqrt{2}}{\sqrt{20-12\sin(2\alpha)}}\right| \leq 1$$
which is equivalent to
$$\left| \frac{\sqrt{2}}{\sqrt{5-3\sin(2\alpha)}}\right| \leq 1$$
Note that $5-3\sin(2\alpha) \geq 2$