0
$\begingroup$

I've taken linear algebra about a year ago, and I don't quite remember everything we've learnt. I was asked a question that given D has a minimum eigenvalue $u$ that is strictly positive, and matrix $D=A^T A$, where A is a matrix of dimension m x n with m >n, what can we say about $u^{T} D u / u^{T}u$ for all $u \neq 0$.

I know by definition that if T is a linear operator on vector space V. Then a nonzero vector $v \in V$ is an eigenvector of T if there exists scalar $\lambda$ such that $T(v) = \lambda v$ and $\lambda$ is the eigenvalue. I also know that scalar $\lambda$ is an eigenvalue iff $\det (A - \lambda I_n ) = 0$.

I've been stuck on this with no idea how to start. Any pointers/help would be much appreciated.

  • 0
    Some observations: $A^TA$, for an $A$ that is full rank leads to a symmetric matrix. If $u$ is an $m \times 1 $ vector, then $u^T D u$ is a scalar – 2017-02-01
  • 0
    @carmicheal561 It does not mention if $u^{T}$ is a scalar, just that $u \neq 0$. I'm also not sure how the quotient is defined. Unfortunately, this is all that I got from the question. – 2017-02-01
  • 0
    Surely you must mean $u^TDu/u^Tu$? – 2017-02-01
  • 0
    @ArnoldFrenzy Ohhhh. I did a quick check from my old textbook. You're right, I forgot $A^TA$ results in a symmetric matrix for full rank A. But for A to be full rank, we need linear independence and this can only happen if we either have column independence or row independence? – 2017-02-01
  • 0
    @ZackCramer Oh I'm so terribly sorry. Yes -- I've updated the post. Thanks! – 2017-02-01

2 Answers 2

1

I will take $u$ to be an arbitrary vector, and not an eigenvalue... I will assume the minimum eigenvalue of $D$ is positive, and call it $\lambda_{\min}$.

$D$ is symmetric regardless of the rank of $A$.

Therefore it is diagonalizable as $D=BAB^\top$, where $A$ is diagonal with positive entries on the diagonal (the eigenvalues) and $B$ is orthogonal.

Then for any vector $u$, $$\frac{u^\top D u}{u^\top u} = \frac{(Bu)^\top A (Bu)}{u^\top u} \ge \frac{\lambda_{\min} (Bu)^\top (Bu)}{u^\top u} = \lambda_{\min}.$$ The last equality is due to orthogonality of $B$ (that is, $B^\top B=I$).

  • 0
    Sorry, I made quite a mess in this post. The question says that D has minimum eigenvalue $u$ that is strictly positive. Could this be a typo? – 2017-02-01
1

Consider a $n \times n$ symmetric matrix $B$ and $n$-dimensional vector $u \neq 0$. In this situation $\frac{u^T B u}{u^T u}$ is called a Rayleigh quotient. Through orthogonal diagonalization we find that this is $\sum_{i=1}^n c_i \lambda_i$ where the $c_i$ are nonnegative and sum to $1$. In other words a Rayleigh quotient is a weighted average of the eigenvalues of $B$. Moreover if $u$ is actually an eigenvector with eigenvalue $\lambda$ then the Rayleigh quotient is just $\lambda$.

In your case the $B$ is $A^T A$. Such a $B$ is automatically symmetric with nonnegative eigenvalues. Additionally it has strictly positive eigenvalues if and only if $A$ has full column rank (i.e. it is $m \times n$ with rank $n$).

  • 0
    Thank you! It seems I have a lot more brushing up to do this, but this definitely helped a lot. – 2017-02-01