0
$\begingroup$

I was reading this article related to autoregressive processes of order $1$. According to wiki it is given by

$ x_t = \phi{x_{t-1}} + \epsilon \\ |\phi| < 1 \\ x_t|x_1,\ldots,x_{t-1} \sim N(\phi{x_{t-1},1}) $

I didn't get what this function $\phi$ is and how come the conditional distribution has mean equal to $\phi{x_{t-1}}$

Also I was referring to this book Gaussian Markov Random Fields Theory and Applications and they have assumed that the marginal distribution of $x_1$ is normal with mean zero and variance $\frac1{1-\phi^2}$ and modeled the joint density of $x$,

$\pi(x)=\pi(x_1)\pi({x_2|x_1})\cdots\pi(x_n|x_{n-1})$ to be gaussian distribution with precision matrix given by

$Q = ( (1) (-\phi) \cdots\cdots)$ $( (-\phi) ({1 + \phi^2}) (-\phi)\cdots\cdots)$ and so on I didn't get how this precision matrix was obtained. Any clarifications?

1 Answers 1

2

I didn't get what this function $\phi$ is

It's not a function, just a fixed number.

is and how come the conditional distribution has mean equal to $\phi{x_{t-1}}$

Because, assuming $\epsilon_t$ is a iid $N(0,1)$ (white gaussian noise) :

$E(x_t |x_{t-1} ) = E(\phi x_{t-1} + \epsilon_t |x_{t-1}) = \phi x_{t-1} + E(\epsilon_t) = \phi x_{t-1}$

Further, assuming $x_t$ is weakly stationary, so that $E(x_t^m)=E(x^m_{t-1})$, we have

$E(x_t) = \phi E(x_{t-1}) + E(\epsilon_t) \Rightarrow E(x_t) (1-\phi) = 0 \Rightarrow E(x_t)=0$

and squaring both sides:

$E(x_t^2) = E(\phi x_{t-1}+ \epsilon_t)^2 = \phi^2 E( x^2_{t-1}) + E(\epsilon_t^2) $

(the cross terms vanishes because $\epsilon_t$ is independent of $x_{t-1}$) and so the variance of $x_t$ is given by

$E(x_t^2) (1 -\phi^2) = E(\epsilon_t^2) \Rightarrow E(x_t^2) = \frac{1}{1 -\phi^2} $

Added: Here's a derivation for the joint distribution. Let's take $n=4$ , and $a = \phi$ (less typing). Then

$p(x_1 x_2 x_3 x_4 ) = p(x_4 |x_3) p(x_3 |x_2) p(x_2 |x_1) p(x_1)$

The first factor is $N(a \, x_3, 1)$ and the same for the rest, until the last which is $N(0,1/(1-a^2))$, hence

$ p(x_1 x_2 x_3 x_4 ) = \frac{1}{\left(\sqrt{2 \pi}\right)^4 } \exp{\left[ -\frac{1}{2} \left( (x_4- a x_3)^2 + (x_3- a x_2)^2 + (x_2- a x_1)^2 + (1-a^2) \, x_1^2\right)\right]} $

The term inside parentheses can be expanded as

$x_4^2 - 2 a x_3 x_4 + (1+a^2) x_3 - 2 a x_3 x_2 + (1+a^2) x_2 - 2 a x_2 x_1 + x_1^2$

or, in matricial form:

$ \begin{pmatrix} x_1 & x_2 & x_3 & x_4 \\ \end{pmatrix} \begin{pmatrix} 1 & -a & 0 & 0 \\ -a & 1 +a^2 & -a & 0 \\ 0 & -a & 1 +a^2& -a \\ 1 & 0 & -a & 1 \end{pmatrix} \begin{pmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \\ \end{pmatrix} $

So, finally $p({\bf x}) = \frac{1}{\left(\sqrt{2 \pi}\right)^4} \exp{\left( -\frac{1}{2} {\bf x^T Q x } \right)}$

which is the formula a joint 4-D gaussian, with zero mean and covariance $\Sigma = Q^{-1}$ And so we have found the precision matrix (inverse of the correlation matrix).

  • 0
    Fixed the first issue (actually, that's the correct way). And expanded on the second. Regarding "how it is derived in the book", I don't know, I haven't it.2012-06-26