0
$\begingroup$

Suppose $\mathbf A$ is an $n \times r$ matrix and $\mathbf D$ a diagonal $r \times r$ matrix with all elements in the diagonal strictly positive. Also $\mathbf A\mathbf D\mathbf A'$ is invertible. I need to show that

$$ \mathbf D^{-1}-\mathbf A'(\mathbf A\mathbf D\mathbf A')^{-1}\mathbf A $$

is positive semi-definite. I am not sure where to start from though. Any hints would be very welcome.

  • 0
    I would start with the definition and try to simplify $u^T M u$ where $M$ is your complicated expression above. – 2017-02-06
  • 0
    @hardmath Thanks. I tried this, but it didn't really go anywhere... – 2017-02-06
  • 0
    That in turn is an early indication that you are trying to prove something that isn't true, and that attention should shift to trying examples. I note that when $A$ itself is square and invertible, the expression simplifies to zero, which is a "positive semi-definite" matrix technically. – 2017-02-06
  • 0
    We need $n \le r$ and $A$ of full rank in order for $ADA'$ to be invertible. – 2017-02-06
  • 0
    @hardmath I've tried many examples and haven't found a counter-example yet. – 2017-02-06
  • 0
    Notice that in its current form the Question does not assume $D$ is positive definite. – 2017-02-06
  • 0
    @hardmath You are right about that. I forgot to add that assumption, I'll do it now. – 2017-02-06

2 Answers 2

2

Per hardmath's suggestion: Let $y = D^{-1/2}x$, and let $B = AD^{1/2}$. We can rewrite $$ D^{-1} - A'(ADA')^{-1}A = \\ D^{-1/2}(I - D^{1/2}A'(AD^{1/2}D^{1/2}A')^{-1}AD^{1/2})D^{-1/2} =\\ D^{-1/2}(I - B'(BB')^{-1}B)D^{-1/2} $$ To show that $I - B'(BB')^{-1}B$ is positive definite, it suffices to note that $B'(BB')^{-1}B$ defines the orthogonal projection onto the row-space of $B$. In particular, $B'(BB')^{-1}B y$ produces $B'x$, where $x$ solves the "least squares" equation $BB'x = By$. So, $I - B'(BB')^{-1}B$ defines the projection onto the orthogonal complement of the row-space.

For a more computational ending: $M = B'(BB')^{-1}B$ is symmetric with $M^2 = M$. So, its eigenvalues are in $\{0,1\}$. So, the eigenvalues of $I - M$ are also in $\{0,1\}$. So, $I - M$ is symmetric with non-negative eigenvalues.

1

Thoughts

  • We know that $A$ is of full (row) rank since $ADA'$ is invertible. So, there exists a $B$ with $AB = I_{r \times r}$.

  • This manipulation of the inequality

$$ x'\mathbf D^{-1}x-x'\mathbf A'(\mathbf A\mathbf D\mathbf A')^{-1}\mathbf Ax \geq 0 \iff \\ x'\mathbf D^{-1}x\geq x'\mathbf A'(\mathbf A\mathbf D\mathbf A')^{-1}\mathbf Ax \iff \\ x'\mathbf D^{-1}x\geq (\mathbf A x)'(\mathbf A\mathbf D\mathbf A')^{-1}(\mathbf Ax) \iff\\ \frac{(\mathbf A x)'(\mathbf A\mathbf D\mathbf A')^{-1}(\mathbf Ax)}{x'\mathbf D^{-1}x} \leq 1 \\ $$

  • 1
    With the added assumption that $D$ is positive definite, we can "absorb" the square root factors of $D$ into $A,A'$ and thereby reduce consideration to the cases $D=I$. – 2017-02-06
  • 1
    @hardmath I can't believe I didn't think of that! That completely solves it. – 2017-02-06