0
$\begingroup$

I was reading this article related to autoregressive processes of order $1$. According to wiki it is given by

$$ x_t = \phi{x_{t-1}} + \epsilon \\ |\phi| < 1 \\ x_t|x_1,\ldots,x_{t-1} \sim N(\phi{x_{t-1},1}) $$

I didn't get what this function $$\phi$$ is and how come the conditional distribution has mean equal to $$\phi{x_{t-1}}$$

Also I was referring to this book Gaussian Markov Random Fields Theory and Applications and they have assumed that the marginal distribution of $x_1$ is normal with mean zero and variance $$\frac1{1-\phi^2}$$ and modeled the joint density of $x$,

$$\pi(x)=\pi(x_1)\pi({x_2|x_1})\cdots\pi(x_n|x_{n-1})$$ to be gaussian distribution with precision matrix given by

$$Q = ( (1) (-\phi) \cdots\cdots)$$ $$( (-\phi) ({1 + \phi^2}) (-\phi)\cdots\cdots)$$ and so on I didn't get how this precision matrix was obtained. Any clarifications?

1 Answers 1

2

I didn't get what this function $\phi$ is

It's not a function, just a fixed number.

is and how come the conditional distribution has mean equal to $\phi{x_{t-1}}$

Because, assuming $\epsilon_t$ is a iid $N(0,1)$ (white gaussian noise) :

$E(x_t |x_{t-1} ) = E(\phi x_{t-1} + \epsilon_t |x_{t-1}) = \phi x_{t-1} + E(\epsilon_t) = \phi x_{t-1}$

Further, assuming $x_t$ is weakly stationary, so that $E(x_t^m)=E(x^m_{t-1})$, we have

$$E(x_t) = \phi E(x_{t-1}) + E(\epsilon_t) \Rightarrow E(x_t) (1-\phi) = 0 \Rightarrow E(x_t)=0$$

and squaring both sides:

$$E(x_t^2) = E(\phi x_{t-1}+ \epsilon_t)^2 = \phi^2 E( x^2_{t-1}) + E(\epsilon_t^2) $$

(the cross terms vanishes because $\epsilon_t$ is independent of $x_{t-1}$) and so the variance of $x_t$ is given by

$$E(x_t^2) (1 -\phi^2) = E(\epsilon_t^2) \Rightarrow E(x_t^2) = \frac{1}{1 -\phi^2} $$

Added: Here's a derivation for the joint distribution. Let's take $n=4$ , and $a = \phi$ (less typing). Then

$$p(x_1 x_2 x_3 x_4 ) = p(x_4 |x_3) p(x_3 |x_2) p(x_2 |x_1) p(x_1)$$

The first factor is $N(a \, x_3, 1)$ and the same for the rest, until the last which is $N(0,1/(1-a^2))$, hence

$$ p(x_1 x_2 x_3 x_4 ) = \frac{1}{\left(\sqrt{2 \pi}\right)^4 } \exp{\left[ -\frac{1}{2} \left( (x_4- a x_3)^2 + (x_3- a x_2)^2 + (x_2- a x_1)^2 + (1-a^2) \, x_1^2\right)\right]} $$

The term inside parentheses can be expanded as

$$x_4^2 - 2 a x_3 x_4 + (1+a^2) x_3 - 2 a x_3 x_2 + (1+a^2) x_2 - 2 a x_2 x_1 + x_1^2$$

or, in matricial form:

$$ \begin{pmatrix} x_1 & x_2 & x_3 & x_4 \\ \end{pmatrix} \begin{pmatrix} 1 & -a & 0 & 0 \\ -a & 1 +a^2 & -a & 0 \\ 0 & -a & 1 +a^2& -a \\ 1 & 0 & -a & 1 \end{pmatrix} \begin{pmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \\ \end{pmatrix} $$

So, finally $$p({\bf x}) = \frac{1}{\left(\sqrt{2 \pi}\right)^4} \exp{\left( -\frac{1}{2} {\bf x^T Q x } \right)}$$

which is the formula a joint 4-D gaussian, with zero mean and covariance $\Sigma = Q^{-1}$ And so we have found the precision matrix (inverse of the correlation matrix).

  • 0
    What about the precision matrix. How was it derived? I am referring to this book http://books.google.com/books?id=TLBYs-faw-0C&printsec=frontcover&source=gbs_ge_summary_r&cad=0#v=onepage&q&f=false2012-06-25
  • 0
    Its just a matter of writing the joint gaussian as a product of conditionals. You're supposed to know already about multidimensional gaussian variables, and what the precision matrix is. Doesn't the book explain it?2012-06-25
  • 0
    Yeah I got what the precision matrix and multidimensional gaussian variables are. But what I didn't get is how come that we have the value like phi and 1+square(phi) in the precision matrix for the variables or the nodes that are neighbors to each other. I got the idea of conditionally independence thing. I am just concerned how the value arose2012-06-25
  • 0
    added, see if you get that2012-06-25
  • 0
    Thanks for the derivation. However, I have one more question. In the book, the distribution of x1 is given as N(0,1/(1-phi^2)). But you have taken it to be N(0,1). Also it can be seen when you derived the variance of xt. Also I didn't get how to derive the covariance matrix sigma. It is inverse of precision matrix definitely. But how is it derived in the book2012-06-26
  • 0
    Fixed the first issue (actually, that's the correct way). And expanded on the second. Regarding "how it is derived in the book", I don't know, I haven't it.2012-06-26