1
$\begingroup$

Assume first $A$ has independent columns. We have a projection matrix $P$ on the range of $A$, i.e. $P_A = A(A^\top A)^{-1}A^\top$. Consider now a matrix $B$ which has the same range as $A$. We have $B = AC$, where $C$ is an invertible matrix. We also have the following.

$P_B = B(B^\top B)^{-1}B^\top = AC(C^\top A^\top AC )^{-1}C^\top A^\top = ACC^{-1}(A^\top A)^{-1} {C^{\top}}^{-1}C^\top A = P_A$


I am interested in having a similar result for the case where A does not have independent columns. Now, $A^\top A$ isn't invertible and the projection matrix becomes $P_A = A(A^\top A)^{\dagger}A^\top$, where we denote by $(\cdot)^\dagger$ the Moore-Penrose pseudoinverse. Consider now two cases:

  1. $B$ also has dependent columns and $B = AC$ for some invertible $C$.
  2. $B$ has independent columns and $B = AC$ for a skinny rectangular matrix $C$ such that $C^\top C = I$, i.e. $C$ selects a minimal subset of columns of $A$ to span the range. Edit: The columns of C are columns of the identity matrix.

In each case, I would like a proof that $A(A^\top A)^{\dagger}A^\top = B(B^\top B)^{\dagger}B^\top$. The problem with the above technique is that I can't simply pull $C$ out of $(C^\top A^\top AC )^{\dagger}$.


Edit: I see that the matrix $A(A^\top A)^{\dagger}A^\top = A A^\dagger$, but again, I don't see how to show that $A A^\dagger = AC (AC)^\dagger$.


Edit: I can also see the following justification: $P_A$ is the projection matrix on the range of $A$, and $P_B$ on the range of $B$. Since the range is the same, they also have to be the same by uniqueness of projection (@martini). But this is ugly, because we have to interpret the matrices as projections. Isn't there an $algebraic$ way to justify this?

  • 0
    @francis-jamet, thanks for your reply. By $M^\dagger$ I mean the Moore-Penrose pseudoinverse. I now edited the question to make this clear.2012-10-04

1 Answers 1

1

If $A^{\top}$ is the transpose of $A$, we may have $A \neq 0$ and $P_A=0$. So, $P_A$ is not the projection on the range of $A$ in this case.

Example: If $A=\left( \matrix {1 & 1+i \\ i &i-1} \right)$, $A^{\top} A=0$, so $(A^{\top}A)^{\dagger}=0$ and $P_A=0$, but $A \neq0$.

So, we suppose $A^{\top}$ is the Hermitian transpose (or conjugate transpose). We have $P_A P_A=P_A$ and $P_A^{\top}=P_A$.

  1. If $B=AC$ with $C$ invertible. We have $A^{\top}A(A^{\top}A)^{\dagger}A^{\top}A=A^{\top}A$. So, $P_B P_A P_B = B(B^{\top}B)^{\dagger}C^{\top}A^{\top}P_A AC (B^{\top}B)^{\dagger}B $ $=B(B^{\top}B)^{\dagger}C^{\top}A^{\top}AC (B^{\top}B)^{\dagger}B=P_B P_B=P_B $

As $A=BC^{-1}$, we have $P_A P_B P_A=P_A$ too.

If $M$ is a matrix, $M^{\top}M=0 \implies M=0$.

If $M=(Id-P_B)P_A$, $M^{\top}M=P_A(Id-P_B)P_A=P_A-P_A P_BP_A=0$, so $M=0$ and $P_A-P_BP_A=0$, and $P_A=P_B P_A$. By hermitian transposition, $P_A=P_A P_B$.

We have $P_B=P_A P_B=P_B P_A$, so $P_A=P_B$.

  • 0
    Thank you for the excellent answer. Actually, this also solves point 2, since invertability of $C$ isn't required. Because the span of $A$ and $B$ is the same, we can write $A(A^\top A)^\dagger A^\top = ACX = BX$ for some $X$. Then $P_A P_B P_A = X^\top B^\top B (B^\top B)^\dagger B^\top B X = X^\top B^\top B X = P_A P_A = P_A$.2012-10-08