We review the (correct) procedure that you went through. We have $12^2+9^2=15^2$, so we rewrite our expression as
$$15\left(\frac{12}{15}\cos a -\frac{9}{15}\sin a\right).$$
Now if $b$ is any angle whose cosine is $\frac{12}{15}$ and whose sine is $\frac{9}{15}$, we can rewrite our expression as
$$15\left(\cos a \cos b -\sin a \sin b\right),$$
that is, as $15\cos(a+b)$.
The maximum value of the cosine function is $1$, and the minimum is $-1$. So the maximum and minimum of our expression are $15$ and $-15$ respectively. The only remaining problem is to decide on the appropriate values of $a$.
For the maximum, $a+b$ should be (in degrees) one of $0$, $360$, $-360$, $720$, $-720$, and so on. The angle $b$ is about $36.87$ plus or minus a multiple of $360$. So we can get the desired kind of sum $a+b$ by choosing $a\approx 360-36.87$, about $323.13$.
It is not hard to do a partial verification our answer by calculator. If you compute $12\cos a -9\sin a$ for the above value of $a$, you will get something quite close to $15$. The book's value gives something smaller, roughly $14.4$. The book's value is mistaken. It was obtained by pressing the wrong button on the calculator, $\sin^{-1}$ instead of $\cos^{-1}$.
For the minimum, we want $a+b$ to be $180$ plus or minus a multiple of $360$. Thus $a$ is approximately $180-36.87$.