1
$\begingroup$

Let $P$ be the transition matrix for a Markov chain. I understand what it means.

But what information can I get from matrix $P^2$, or $P^{\infty}$?

As I understand it now, $P^2$ holds the probabilities of getting from initial state $i$ (rows of matrix) and finishing in state $j$ (columns of matrix) in exactly 2 transitions.

So for $P^2 = \begin{pmatrix} 0.1 & 0.9 \\ 0.2 & 0.8 \\ \end{pmatrix}$, where states are $1,2$

There's 10% chance that if I begin in state 1, I will end up in state 1 in exactly two transitions; and 90% that I will end up in state 2.

Do I understand this correctly?

Thank you in advance.

  • 2
    Yes. You can derive that easily using the law of total probability and matrix multiplication. – 2017-01-22
  • 2
    Yes again, slightly more elaborately: Let $p_{ij}(k)$ be the probability of a transition from $i$ to $j$ in $k$ steps. Then the the upper left element of $P^2$ is $p_{00}(2) = p_{00}(1)p_{00}(1) + p_{01}(1)p_{10}(1),$ where $p_{ij}(1) = p_{ij}$ are elements of $P.$ This equation can be proved using conditional probabilities and the Markov property; it is also the same as the formula for the upper-left element of $P^2$ by matrix multiplication. – 2017-01-23

0 Answers 0