2
$\begingroup$

I am following this article on face recognition. In "calculating eigenfaces" section, the authors present a solution for the problem of calculating a very big matrix:

Let $A_{N^2\times M}$ be an $M$ sized dataset, where each column in an $N\times N$ image. Instead of calculating the $M$ large eigenvectors of the $N^2\times N^2$ co-variance matrix they calculate the $M$ eigenvectors of $L=A^TA$ matrix which is of size $M\times M$.

  • Why is this a valid\good enough solution?
  • What are the criteria for a largest vector? larger in which seance?
  • 2
    Questions like these are yet another reason why I prefer the SVD presentation of these results as opposed to the eigenvalue versions. – 2017-01-07

1 Answers 1

4

If $A$ and $B^T$ are $m\times n$ matrices (so both products $AB$ and $BA$ exist) then their characteristic polynoms $p_{AB}(\lambda)$ and $p_{BA}(\lambda)$ are almost the same (look for the sketch of proof at Wikipedia): $$ p_{AB}(\lambda)=(-\lambda)^{m-n}p_{BA}(\lambda). $$ Therefore, their non-zero eigenvalues are the same as well.

Also, for any eigenvector $x$ of matrix $AB$ corresponding to a non-zero eigenvalue $\lambda$ the vector $Bx$ becomes an eigenvector of $BA$ and corresponds to the same eigenvalue. $$ (BA)(Bx)=B(AB)x=B(\lambda x)=\lambda(Bx). $$

  • 0
    Could you add some more detail on how this leads to the OP's result? – 2017-01-07
  • 0
    @nbubis Ah, sorry, I've concentrated on eigenvalues and forgot to add a note on eigenvectors. Will do in a minute. – 2017-01-07
  • 0
    @SergeiGolovan How long are your minutes :D. If possible I would still like further clarification on the eigenvectors. – 2018-01-09
  • 1
    @Shinobii What do you need to be clarified? – 2018-01-10
  • 0
    Perhaps I missed the reasoning, but why is it sufficient to find the M x M eigenvectors? I understand the math, just having a hard time explaining it to others. I suppose my question is, why are there only M meaningful eigenvectors in the $N^2$ x $N^2$ matrix? – 2018-01-10
  • 1
    @Shinobii: The other eigenvalues are zero. Basically, since the matrix in question is symmetric, the orthogonal complement of the $M$ eigenvectors' span is the nullity of the matrix. – 2018-01-10