2
$\begingroup$

Define

$ \mathbf{H}=\mathbf{X}\left(\mathbf{X}^{\prime}\mathbf{X}\right)^{-1}\mathbf{X}^{\prime} $

where $\mathbf{X}$ is of order $n \times k$

and

$ \overline{\mathbf{J}}=\frac{1}{n}\mathbf{J}=\frac{1}{n}\mathbf{1}\mathbf{1}^{\prime} $

where $\mathbf{1}$ is a unit vector of order $n \times 1$.

Now $ \mathbf{H}\mathbf{H}=\mathbf{H} $

and

$ \overline{\mathbf{J}}\overline{\mathbf{J}}=\overline{\mathbf{J}} $

Thus both $\mathbf{H}$ and $\overline{\mathbf{J}}$ are idempotent matrices.

My question is whether $\mathbf{H}-\overline{\mathbf{J}}$ would be idempotent. If so then

$ \left(\mathbf{H}-\mathbf{\overline{\mathbf{J}}}\right)\mathbf{\left(\mathbf{H}-\mathbf{\overline{\mathbf{J}}}\right)}=\mathbf{H}-\mathbf{H\overline{\mathbf{J}}-\overline{\mathbf{J}}H}+\overline{\mathbf{J}}=\mathbf{H}-\mathbf{\overline{\mathbf{J}}-\overline{\mathbf{J}}}+\overline{\mathbf{J}}=\mathbf{H}-\overline{\mathbf{J}} $

But I'm not able to show that

$ \mathbf{H\overline{\mathbf{J}}=\overline{\mathbf{J}}H}=\overline{\mathbf{J}} $

I'd highly appreciate if you guide me to figure this. Thanks for your time and help.

  • 0
    Andreas Spörl: It is not enough for them to commute. E.g., the difference is not idempotent when $\mathbf X$ is $n$-by-$1$ and orthogonal to $\mathbf 1$, even though $\mathbf{H}$ and $\overline{\mathbf{J}}$ commute in that case.2012-01-05

3 Answers 3

2

I'm assuming your scalars are real. I'm not sure what would happen over an arbitrary field.

Here $H$ and $J$ are $n \times n$, but presumably $X$ is $n \times m$ where $n > m$. Try an example with almost any $3 \times 2$ matrix $X$.

In general, $H$ is the orthogonal projection on the column space of $X$, while $\overline{J}$ is the orthogonal projection on the span of $\bf 1$. If $\bf 1$ is in the column space of $X$, then $H - \overline{J}$ is indeed an idempotent, the orthogonal projection on the orthogonal complement of $\bf 1$ in the column space of $X$. If not, then {\bf 1}' (H - \overline{J}) {\bf 1} < 0, which is impossible for an orthogonal projection (and any real symmetric idempotent is an orthogonal projection).

EDIT: over an arbitrary field whose characteristic does not divide $2n$, it's still true. Of course for $\overline{J}$ to exist, the characteristic can't divide $n$. A more general statement is: if $H$ and $K$ are symmetric idempotent matrices over a field not of characteristic $2$, then $H - K$ is an idempotent if and only if the column space of $K$ is contained in the column space of $H$.

If $H$ and $K$ are idempotents, $H - K$ is an idempotent if and only if $HK + KH = 2K$. Multiplying on the left by $K$, we get $KHK + KH = 2K$, so $HK = KHK$, and similarly multiplying on the right by $K$ we get $KH = KHK$. Thus $HK + KH = 2KHK = 2K$, so $HK = KH = K$ (here's where the characteristic can't be $2$). Thus the column space of $K$ is contained in the column space of $H$.

Conversely, if the column space of $K$ is contained in the column space of $H$, then $H K = K$ (i.e. for every vector $v$, $Kv = Hw$ for some $w$, and then $H K v = H H w = H w = K v$). If H' = H and K' = K, this implies K H = K' H' = (H K)' = K' = K, and so $H - K$ is an idempotent.

  • 0
    (+1) Thanks @Robert for your elaborate answer. Much appreciated.2012-01-06
1

${Added}$

I hastily assumed $\mathbf{X}$ to be a square invertible matrix before the question was edited. So this answer works only for that case.

Given \mathbf{H}=\mathbf{X}(\mathbf{X'}\mathbf{X})^{-1}\mathbf{X'}.

Thus, \mathbf{H}=\mathbf{X}\mathbf{X^{-1}}\mathbf{(X')^{-1}}\mathbf{X'}=\mathbf{I} and $\mathbf{I}$ commutes with any matrix, thus it also commutes with $\bar{{\mathbf{J}}}$.

Hence, $\mathbf{H}\bar{{\mathbf{J}}}=\bar{{\mathbf{J}}}\mathbf{H}=\bar{{\mathbf{J}}}$ and we are done.

This works fine iff $\mathbf{X}$ is a square matrix- thanks to Jonas Meyer who pointed out the limitations of my earlier claim.

  • 0
    @JonasMeyer Grr got the hitch now. Thanks for the clarification.2012-01-05
1

Added: Before the question edit, I hastily assumed that $\mathbf X$ is a nonzero column vector. So the answer at bottom only concerns the case where $k=1$. In the general case, it holds if $\mathbf 1$ is in the column space of $\mathbf X$ (see Robert Israel's answer).


In general, if $P$ and $Q$ are self-adjoint projections on an inner product space $V$, then $P-Q$ is a projection if and only if $QV\subseteq PV$ if and only if $PQ=QP=Q$.


Consider the special case $k=1$. Then it is true if and only if $\mathbf{X}$ is a nonzero scalar multiple of $\mathbf{1}$ (which I presume is the column vector with all $1$s). If $\mathbf X=\lambda\mathbf 1$ for some $\lambda\neq 0$, then $\mathbf H=\overline{\mathbf J}$, so $\mathbf H-\overline{\mathbf J}=0$.

Suppose that $\mathbf X$ is not a scalar multiple of $\mathbf 1$. Then $\|\langle \mathbf 1,\mathbf X\rangle\|<\|\mathbf 1\|\|\mathbf X\|$. Therefore $\|\mathbf{H}\overline{\mathbf{J}}\mathbf1\|=\|\mathbf H\mathbf 1\|=\left\|\frac{\langle \mathbf 1,\mathbf X\rangle}{\|\mathbf X\|^2}\mathbf X\right\|<\|\mathbf 1\|$. This implies that $\mathbf H\overline{\mathbf J}+\overline{\mathbf J}\mathbf H\neq 2\overline{\mathbf J}$.

  • 0
    Hm. I didn't notice that either. Sorry for my comment.2012-01-05