Condition number inequality proof with perturbed matrix

Question

There is this exercise in a numerical course book in the topic of regularization of the least squares approximation, which goes like this:

Prove that for every $A\in R^{n\times m}$ real or complex matrix $$\kappa_2^2(A) \geq \kappa_2(A^TA + \gamma I),$$ where $\gamma>0$ is a real number, and $I$ is the appropriate size identity matrix.

Can you help me with this proof?

I have proved that $\kappa_2^2(A) = \kappa_2(A^TA)$, but in the original exercise I can't give a bound on $||(A^TA + \gamma I)^{-1}||$. I'm curious about this bound also, even if the exercise can be solved without it.

Answer 1

For a symmetric positive definite matrix $B$, $\kappa_2(B)=\|B\|_2\|B^{-1}\|_2=\lambda_\max(B)/\lambda_\min(B)$. We have $$ \kappa_2^2(A)=\kappa_2(A^TA)=\frac{\lambda_\max(A^TA)}{\lambda_\min(A^TA)} $$ and $$ \kappa_2(A^TA+\gamma I)=\frac{\lambda_\max(A^TA+\gamma I)}{\lambda_\min(A^TA+\gamma I)}=\frac{\lambda_\max(A^TA)+\gamma}{\lambda_\min(A^TA)+\gamma}. $$ The only thing you need to show now is that for $\alpha\geq\beta>0$, the function $$ \gamma\mapsto\frac{\alpha+\gamma}{\beta+\gamma} $$ is non-increasing for $\gamma>0$.