3
$\begingroup$

I read in this answer that:

If covariance matrix is $\Sigma$, the covariance after projecting in $u$ is $u^T \Sigma u$.

I fail to see this, how do I get the covariance of a set of points after projecting those points along the direction $u$ as a function of $u$ and $\Sigma$ ?

  • 3
    More generally, if $X\in\mathbb{R}^n$ and $Y\in\mathbb{R}^m$ are random vectors and $\operatorname{cov}(X,Y)=\Sigma\in\mathbb{R}^{n\times m}$, and $A\in\mathbb{R}^{k\times n}$ and $B\in\mathbb{R}^{m\times\ell}$ are constant (i.e. non-random) matrices, then $\operatorname{cov}(AX,BY)=A\Sigma B^T\in\mathbb{R}^{k\times \ell}$. More tersely, $\operatorname{cov}(AX,BY)=A(\operatorname{cov}(X,Y))B^T$.2012-07-23

1 Answers 1

6

The covariance matrix for a vector quantity $x$ is $\langle xx^\top\rangle-\langle x\rangle\langle x^\top\rangle$. The covariance for the projection $u^\top x$ is

$\langle u^\top xx^\top u\rangle-\langle u^\top x\rangle\langle x^\top u\rangle=u^\top\langle xx^\top\rangle u-u^\top\langle x\rangle\langle x^\top\rangle u=u^\top\left(\langle xx^\top\rangle-\langle x\rangle\langle x^\top\rangle\right)u\;.$

The point is basically that you can pull $u$ out of all the expectation values because it's a constant.

  • 0
    @joriki makes sense. However, allow me to be nit-picky by pointing out that your answer seems to use the formula for covariance of a vector $x$, but $u^\top x$ is a scalar. You should rather use $Cov(s) = \langle s^2 \rangle - \langle s \rangle\langle s \rangle$ for a scalar $s$, which again reduces to $\langle u^\top xx^\top u\rangle-\langle u^\top x\rangle\langle x^\top u\rangle$ for $u^\top x$ because $(u^\top x)^2=(u^\top x)(x^\top u)$2016-01-07