Finding the expectation of $E(T)$ of a $4 \times 4$ Markov Chain $T = \min\{ n \geq 1 : X_n = 3\}$?

Question

Given the following Markov Chain:

$$M = \left( \begin{array}{cccc} a_0 & a_1 & a_2 & a_3 \\ 0 & a_0+a_1 & a_3 & a_2 \\ 0&0 & a_0+a_1+a_3 & a_2\\ a_3 &a_2 & a_1 & a_0 \\ \end{array} \right)$$

where all $a_i >0$ and $a_0+a_1+a_2+a_3 = 1$. The Markov chain starts from $X_0 = 0$. Define $T = \min\{ n \geq 1 : X_n = 3\}$, the first time the chain hits state $3$. I am trying to find the expectation of $T$.

So far, I have been doing first step analysis. Let $v_i = E(T\mid X_0 = i)$. Then, we have the equations:

\begin{align} v_0 &= 1+a_0v_0 + a_1v_1 + a_2v_2 + a_3v_3 \\ v_1 &= 1+(a_0+a_1)v_0 + a_ev_2 + a_3v_3 \\ v_2 &= 1+(a_0+a_1+a_3)v_2 + a_2v_3 \\ v_3 &= a_3v_0 + a_2v_1+a_2v_1 \end{align}

Using the fact $v_3 = 0$, I end up getting three equations in two variables:

\begin{align} (1-a_0)v_0 &= 1+a_1v_1 \\ (1-a_0-a_1)v_1 &= 1 \\ a_3v_0+a_2v_1 &=0\\ \end{align}

which isn't correct. Whats the right way to do this? Thanks!

Answer 1

Based on the given matrix $M$, the correct system is \begin{align} v_0 &= 1+a_0v_0 + a_1v_1 + a_2v_2 + a_3v_3 \\ v_1 &= 1+(a_0+a_1)v_1 + a_3v_2 + a_2v_3 \\ v_2 &= 1+(a_0+a_1+a_3)v_2 + a_2v_3 \\ v_3 &= 0 \end{align}