2
$\begingroup$

Suppose the Gaussian random vector $\mathbf{X}\sim\mathcal{N}(\mathbf{\mu_X},\Sigma_\mathbf{X})$ where $$\mathbf{\mu_X}=\begin{bmatrix}1\\5\\2\end{bmatrix}$$ and $$\Sigma_\mathbf{X}=\begin{bmatrix}1&1&0\\1&4&0\\0&0&9\end{bmatrix}$$ I want to calculate the following PDFs: $$X_3\text{ given }\begin{bmatrix}X_1&X_2\end{bmatrix}^T$$ and $$X_1\text{ given }\begin{bmatrix}X_2&X_3\end{bmatrix}^T$$

For $X_3|\begin{bmatrix}X_1&X_2\end{bmatrix}^T$ we have that $X_3$, $X_1$ are uncorrelated since $\Sigma_\mathbf{X}(1,3)=cov(X_1,X_3)=0$ and thus independent as they are jointly Gaussian. Similarly $X_3$, $X_2$ are independent. So the conditional PDF becomes $$f_{X_3|\begin{bmatrix}X_1&X_2\end{bmatrix}^T}=f_{X_3}\sim\mathcal{N}(\mathbf{e_3^T}\mathbf{\mu_X},\mathbf{e_3^T}\Sigma_\mathbf{X}\mathbf{e_3})=\mathcal{N}(2,9)$$

for $\mathbf{e}_i$ a column vector with $1$ at position $i$ and $0$ elsewhere.

But in the other case of $X_1|\begin{bmatrix}X_2&X_3\end{bmatrix}^T$ even if $X_1$, $X_3$ are independent nevertheless $X_1$, $X_2$ are not and I cannot eliminate any of the r.v.s that appear nor can I imply conditional independence. Any ideas?

EDIT

Since $(X_1,X_2)$ is independent of $X_3$, we have the following conditional PDF formula $$X_1|\{X_2=x_2,X_3=x_3\} = X_1|\{X_2=x_2\}\sim\mathcal{N}(\Sigma_{12}\Sigma_{22}^{-1}(x_2-\mu_2)+\mu_1, \Sigma_{11}-\Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21})$$

  • 0
    In your case, the conditional distribution of $X_1$ given $X_2$ and $X_3$ is the conditional distribution of $X_1$ given $X_2$ only. Did it become easier? – 2017-01-14
  • 0
    @zhoraster I am not sure that this is correct. Independence between two random variables does not necessarily imply conditional independence given a third random variable. – 2017-01-14
  • 0
    Here $X_3$ is independent of *both* $X_1$ and $X_2$. – 2017-01-14
  • 0
    @zhoraster OK, so I have found a formula for the conditional PDF of random vectors. I guess its proof deals with some tricks with the information and covariance representation of Gaussian random vectors. However, I am not sure how to calculate those $\Sigma_{ij}$ matrices. I couldn't find their definition. Can you check the edit above? – 2017-01-14
  • 0
    Well, here these matrices are $1\times 1$, so they're just numbers: $\Sigma_{11} = \Sigma_{12} = 1$, $\Sigma_{22} = 4$. – 2017-01-15
  • 0
    @zhoraster So, I also suppose that $\Sigma_{21}=1$ and thus in this case $X_1|\{X_2=x_2\}\sim\mathcal{N}((x_2-5)/4,3/4)$. Also, I am not sure how matrices $\Sigma_{ij}$ are calculated in general. When, for example, I need to calculate $f_{\mathbf{x|y}}$ for $\mathbf{x,y}$ Gaussian random vectors can you suggest me some material that explains this PDF and how these $\Sigma_{ij}$ matrices are computed? – 2017-01-16
  • 0
    $\Sigma_{i\,j} = \big(\operatorname{Cov}(X_{i\, k},X_{j\, l}); k=1,\dots,\dim X_i, l = 1,\dots,\dim X_j\big)$. For example, for $i=j$ this is just the covariance matrix of $X_i$. – 2017-01-16

0 Answers 0