4
$\begingroup$

I have a Markov chain with state space $E = \{1,2,3,4,5\}$ and transition matrix below:

$$ \begin{bmatrix} 1/2 & 0 & 1/2 & 0 & 0 \\ 1/3 & 2/3 & 0 & 0 & 0 \\ 0 & 1/4 & 1/4 & 1/4 & 1/4 \\ 0 & 0 & 0 & 3/4 & 1/4 \\ 0 & 0 & 0 & 1/5 & 4/5 \end{bmatrix} $$

Given the initial distribution $\pi = (1/2,0,0,1/2,0)$, how would I calculate $\mathbb{P}(X_2 = 4)$? Do I have to multiply $\pi$ by the transition matrix etc.?

1 Answers 1

1

If "initial distribution" means the distribution of $X_1$, then the distribution of $X_2$ is $\pi M$, where $M$ is the transition matrix above.

If "initial distribution" means the distribution of $X_0$, then the distribution of $X_2$ is $\pi M^2$.

  • 0
    Thanks Michael! I think the question means the "initial distribution" means the distribution of $X_0$. Just one quick question, I cant seem to multiply $M^2 \pi$, because the dimensions (row and columns) are not correct, but I can multiply $\pi M^2$, and I get the correct answer $(59/160)$. So the distribution $M^2 \pi$ or $\pi M^2$?2012-04-10
  • 0
    It should be $\pi M^2$. This is a "right stochastic matrix" (rows sum up to 1), which means it's on the "right" of a matrix multiplication. It is my understanding that transition matrices are by convention situated on the right. I'm not sure why this is so; maybe someone else can enlighten us.2012-04-11
  • 0
    Yes, please, can someone enlighten us! @X.Yang, so if I have a left stochastic matrix, would it be on the "left" of a matrix multiplication? Or right?2012-04-11
  • 0
    On the left - that's where the name comes from. A left stochastic matrix is defined as one where all the columns sum up to 1. This is essential because, when you end up doing the matrix multiplication, you'll see that each state has a total probability 1 of going to any other state - which only makes sense. Same thing for right stochastic matrices.2012-04-11
  • 0
    Thanks, what do you mean then when you said that by convention the transition matrices are situation on the right?2012-04-11
  • 0
    I mean that when we write down the stochastic matrices, we usually write them down as right stochastic matrices unless otherwise stated. Sorry for the confusion.2012-04-11
  • 0
    Thanks for your clarification, and no worries :)2012-04-11
  • 0
    I confused the roles of rows and columns, but now I've fixed it.2012-04-11