1
$\begingroup$

Let $\mathbf A$ be a $n\times r$ matrix with $\mathbf A'\mathbf A$ invertible and, for some $u

$$\mathbf B = \begin{bmatrix}\mathbf I_u & \mathbf 0 \\ \mathbf 0 & \mathbf 0\end{bmatrix}_{n\times n},$$

where $\mathbf I_u$ is the $u\times u$ identity matrix.

What can be said about the eigenvalues of $(\mathbf A'\mathbf A)^{-1}(\mathbf A'\mathbf B\mathbf A)$?

I suspect that the eigenvalues always belong to $[-1,1]$, but don't know how start. Any hints would be very welcome.

  • 0
    Perhaps notable: $[A(A'A)^{-1}A]B$ is the product of projections – 2017-02-12

2 Answers 2

1

We can show that the eigenvalues of $M = (A'A)^{-1}(A'BA)$ are always in $[0,1]$. We also know that the number of zero-eigenvalues will always be $(n-u)$.

Claim: $M$ has real, non-negative eigenvalues

Proof: We find that the similar matrix $$ (A'A)^{1/2}M(A'A)^{-1/2} = (A'A)^{1/2}(A'A)^{-1}(A'BA)(A'A)^{-1/2} = (A'A)^{-1/2}(A'BA)(A'A)^{-1/2} $$ is positive semidefinite.

Claim: The eigenvalues of $M$ are smaller than $1$.

Proof: Suppose that $A$ and $B$ are such that the resulting $M$ has an eigenvalue greater than $1$. Note that by Badam's observation, $I - M$ is similar to a $\tilde M$ resulting from some other pair $\tilde A,\tilde B$ (or, more directly: $M$ is similar to a positive semidefinite matrix). However, this implies that the eigenvalues of $I - M$ are non-negative. This implies that the eigenvalues of $M$ are $\leq 1$, as desired.

Given a vector $y$, $My$ is the solution $x$ to the equation $$ A'(Ax) = A'B(Ay) $$ which is the least squares solution to the equation $Ax = BAy$.

We can also write $M = A^+BA$, where $A^+$ denotes the pseudoinverse.

  • 0
    Thank you. This is pretty nice. I also thought the eigenvalues would be in $\{0,1\}$ initially, but numerically I've ruled that out. Your result is exactly what I was looking for though. – 2017-02-12
  • 0
    @mzp I realize now that I was setting the bottom row to be zero every time. Indeed, the $\{0,1\}$ does **not** check out. – 2017-02-12
1

First consider that $BA$ is just $A$ with the bottom $n-u$ rows zeroed out.

Now let $$ C= \begin{bmatrix} 0 & 0 \\ 0 & I_{n-u} \end{bmatrix} $$ Consider that $(A'A)^{-1}(A'BA) + (A'A)^{-1}(A'CA) = I$ by linearity.

Hope that's a helpful start!

  • 0
    Thank you. This seems promising, but would you mind elaborating a bit more? – 2017-02-12