I just started learning about Markov processes and got the following homework question.
Classify all the states as recurrent or transient for the Markov chain below
$\begin{matrix} {}&s_1&s_2&s_3&s_4&s_5\cr s_1&1/4 & 3/4 & 0 & 0 & 0\cr s_2&1/2 & 1/2 & 0 & 0 & 0\cr s_3&0 & 0 & 1 & 0 & 0\cr s_4&0 & 0 & 1/3 & 2/3 & 0\cr s_5&1 & 0 & 0 & 0 & 0 \end{matrix}$
I worked through the first few powers of the matrix and found the following:
For n=2 $\begin{matrix} {}&s_1&s_2&s_3&s_4&s_5\cr s_1&7/16 & 9/16 & 0 & 0 & 0\cr s_2&3/8 & 5/8 & 0 & 0 & 0\cr s_3&0 & 0 & 1 & 0 & 0\cr s_4&0 & 0 & 5/9 & 4/9 & 0\cr s_5&1/4 & 3/4 & 0 & 0 & 0 \end{matrix}$
For n=3 $\begin{matrix} {}&s_1&s_2&s_3&s_4&s_5\cr s_1&25/64 & 39/64 & 0 & 0 & 0\cr s_2&13/32 & 19/32 & 0 & 0 & 0\cr s_3&0 & 0 & 1 & 0 & 0\cr s_4&0 & 0 & 19/27 & 8/27 & 0\cr s_5&7/16 & 9/16 & 0 & 0 & 0 \end{matrix}$
From the diagonals, I can figure out that state 3 is recurrent because $\Sigma p_{33}(n)=\infty$
By the same reasoning, states 4 and 5 are transient because $\Sigma p_{44}(n)$ and $\Sigma p_{55}(n)$ are convergent.
First of all, is this method of finding whether the states are recurrent or transient correct? Is there a simpler way to solve the problem? Second, I can't figure out whether states 1 and 2 are transient or recurrent using this method.
Would appreciate if someone can point me in the right direction. Many thanks!