Is the following inequality satisfied?

Question

I would like to show that the following inequality is satisfied:

$$ \| \bigg( A + \sum_{i = 1}^m c_i c_i^T \bigg)^{-1} e_i \|_2 \leq \| A^{-1} e_i\|_2 $$

if $P \triangleq A^{-1}$ satisfies the following conditions:

1. ‍‍‍‍‍‍ ‍‍All the elements of $P$ are non-negative
2. ‍‍‍‍‍‍ ‍‍$P_{ij} \geq {\rm{min}}(P_{ik},P_{kj}), \ \forall (i,j,k) \in (1,\dots,n)$
3. ‍‍‍‍‍‍ ‍‍$P_{ii} > P_{ik}, \ \forall (i \neq k) \in (1,\dots,n)$

where $A \in \mathbb{R}^{n \times n}$ is a (symmetric) positive definite matrix and $c_i \in \mathbb{R}^{n \times 1}$, which has only one non-zero entry, thus, $c_i = \bar{c}_i . e_j $ and $e_i$ is a unit vector that is equal to one at the $i$-th entry and is zero elsewhere.

I was able to show that the above inequality held when $m = 1$, by applying the Sherman-Morrison formula to the term on the left of the inequality to get the following inequality:

$$ \| \bigg( P - \tilde{c}_j . P (e_j e_j^T)P \bigg) e_i \|_2 \leq \|P e_i\|_2 $$

where $(\bar{c}^{-2} + e_j^T P e_j)^{-1} = \tilde{c}_j \in \mathbb{R}_+$ and $P e_i$ represents the $i$-th column of the matrix $P$. The above inequality can be written component-wise:

$$ | P_{ki} - \bigg( \frac{P_{ij}}{\bar{c}^{-2} + P_{jj}} \bigg) P_{kj} | \leq | P_{ki} |$$

$\forall (i,j,k) \in (1,\dots,n)$. The above can be equivalently re-written as

$$ 0 \leq \frac{P_{ij} P_{kj}}{\bar{c}^{-2} + P_{jj}} \leq 2 P_{ki} \qquad (\star) $$

For property 1.) above, the left hand side of the inequality is satisfied. So, it needs to be shown that the RHS of the above inequality would be satisfied under properties 1-3 above. From property 2.), we have:

$$ P_{ki} \geq {\rm{min}}(P_{kj},P_{ji}), \ \forall (i,j,k) \in (1,\dots,n) \qquad (\star \star) $$

And we consider two cases for the previous inequality:

a. Assume $P_{ji} \leq P_{kj}$, then, re-writing the inequality $(\star)$ as:

$$ 2 P_{ki} \geq P_{ij} \bigg( \frac{ P_{kj}}{\bar{c}^{-2} + P_{jj}} \bigg) \geq P_{ij} $$

where the term in the parenthesis is less than 1 from property 3.) above. From our assumption $P_{ji} \leq P_{kj}$, we have from $(\star \star)$ that $P_{ki} \geq P_{ji} = P_{ij}$ and the equality follows from $P$ being symmetric. Thus, the above inequality is satisfied.

b. Similar argument for $P_{ji} \geq P_{kj}$.

I need help in coming up with a similar inequality as $(\star)$ for $m > 1$. I wanted to use an answer from here - Sherman-Morrison updates for sum of many rank 1 matrices. - for this case. I would appreciate any help. Thank you.

Answer 1

In general, (without your assumptions on $A^{-1}$), this is not true. We can build a counterexample as follows: choose $P,Q$ such that $P - Q$ is positive definite, but $P^2 - Q^2$ is not (a $2 \times 2$ counterexample exists). With an appropriate change of basis, make it so that $e_1^*(P^2 - Q^2)e_1 < 0$. Let $A = Q^{-1}$ and $B = Q^{-1} - P^{-1}$. We have $$ \left\| \bigg( A + \sum_{i = 1}^m c_i c_i^T \bigg)^{-1} e_1 \right\|_2 > \| A^{-1} e_1\|_2 $$ contrary to your hypothesis.

---

Let $B = \sum c_ic_i^T$. Both $A$ and $B$ are positive definite. Note that for all $x$, we have $$ (A + B) \geq A $$ which is to say that $(A+B) - A$ is positive semidefinite. It follows that $$ A^{-1/2}(A+ B)A^{-1/2} \geq A^{-1/2}AA^{-1/2} = I $$ which is to say that $A^{-1/2}(A+ B)A^{-1/2}$ is positive definite with eigenvalues greater than $1$. It follows that $$ A^{1/2}(A+ B)^{-1}A^{1/2} \leq I $$ Finally, we conclude $$ (A + B)^{-1} \geq A^{-1} $$ which is to say that $(A + B)^{-1} - A^{-1}$ is positive semidefinite. It follows that $$ e_i^*[(A + B)^{-1} - A^{-1}]e_i \geq 0 \implies\\ {\|(A + B)^{-1/2}e_i\|_2}^2 \geq {\|A^{-1/2}e_i\|_2}^2 $$ Or if you prefer, $[(A + B)^{-1}]_{ii} \geq [A^{-1}]_{ii}$. That's as close as we can get.