Expectation of normal distribution with matrix

Question

Let $\boldsymbol{x}\sim \mathcal{N}(\boldsymbol{\mu},\boldsymbol{\Sigma})$, where $\textbf{x}$ is partitioned as $\textbf{x}=(\textbf{x}_1^{T},\textbf{x}_2^{T})$ and $\boldsymbol{\mu}$ and $\boldsymbol{\Sigma}$ are partitioned conformably. Let $\boldsymbol{y}=\textbf{x}_1-\textbf{Ax}_2$ for some conformable nonrandom matrix $A$. Compute E{$\textbf{y}(\textbf{x}_2-\boldsymbol{\mu})^{T}$}.

The solution is E{$\textbf{y}(\textbf{x}_2-\boldsymbol{\mu_2})^{T}$} = $\boldsymbol{\Sigma_{12}}-\boldsymbol{A\Sigma_{22}}$. But I can't figure out why it should be. Could someone help me out? Thanks in advance.

Answer 1

Since $\boldsymbol \mu$ and $\boldsymbol \Sigma$ are partitioned conformably, we have $\boldsymbol \mu_i = \mathbb{E}\{\boldsymbol x_i\}$ and $\boldsymbol \Sigma_{ij} = \mathbb{E}\{(\boldsymbol x_i-\boldsymbol \mu_i)(\boldsymbol x_j-\boldsymbol \mu_j)^T\}$. Multiplying out, we have

$$ \begin{align} \boldsymbol \Sigma_{ij} &= \mathbb{E}\{\boldsymbol x_i\boldsymbol x_j^T-\boldsymbol \mu_i\boldsymbol x_j^T - \boldsymbol x_i \boldsymbol \mu_j^T+ \boldsymbol \mu_i\boldsymbol \mu_j^T\} \\& = \mathbb{E}\{\boldsymbol x_i\boldsymbol x_j^T\} - \boldsymbol \mu_i \mathbb{E}\{\boldsymbol x_j\}^T - \mathbb{E}\{\boldsymbol x_i\}\boldsymbol \mu_j^T + \boldsymbol \mu_i\boldsymbol \mu_j^T \\& = \mathbb{E}\{\boldsymbol x_i\boldsymbol x_j^T\} - \boldsymbol \mu_i\boldsymbol \mu_j^T - \boldsymbol \mu_i\boldsymbol \mu_j^T + \boldsymbol \mu_i\boldsymbol \mu_j^T \\& = \mathbb{E}\{\boldsymbol x_i\boldsymbol x_j^T\} - \boldsymbol \mu_i\boldsymbol \mu_j^T \end{align}$$

Based this, let us calculate:

$$\begin{align} \mathbb{E}\{\boldsymbol y (\boldsymbol x_2 - \boldsymbol \mu_2)^T\} & = \mathbb{E}\{(\boldsymbol x_1 - \boldsymbol A \boldsymbol x_2) (\boldsymbol x_2 - \boldsymbol \mu_2)^T\} \\ & = \mathbb{E}\{\boldsymbol x_1\boldsymbol x_2^T - \boldsymbol A \boldsymbol x_2 \boldsymbol x_2^T - \boldsymbol x_1\boldsymbol \mu_2^T + \boldsymbol A \boldsymbol x_2 \boldsymbol \mu_2^T\} \\& = \mathbb{E}\{\boldsymbol x_1\boldsymbol x_2^T\} - \boldsymbol A \mathbb{E}\{\boldsymbol x_2\boldsymbol x_2^T\} - \boldsymbol \mu_1\boldsymbol \mu_2^T + \boldsymbol A \boldsymbol \mu_2\boldsymbol \mu_2^T \\& = \underbrace{\mathbb{E}\{\boldsymbol x_1\boldsymbol x_2^T\} - \boldsymbol \mu_1\boldsymbol \mu_2^T}_{\boldsymbol \Sigma_{12}} -\boldsymbol A \underbrace{\left(\mathbb{E}\{\boldsymbol x_2\boldsymbol x_2^T\} - \boldsymbol \mu_2\boldsymbol \mu_2^T\right)}_{\boldsymbol \Sigma_{22}} \\& = \boldsymbol \Sigma_{12} - \boldsymbol A \boldsymbol \Sigma_{22} \end{align} $$

Let me know if any of the steps is unclear.