0
$\begingroup$

Is it possible to have null recurrent states if the number of states is finite? If so, I would appreciate a small example (a 2x2 or 3x3 transition probability matrix would be nice).

1 Answers 1

3

For a finite state space Markov chain, every recurrent state is positive.


Here is a paraphrase of the proof from Probability and Stochastic Processes by Grimmett and Stirzaker. Consider a communicating class $C$ containing a null, recurrent state. Then $p_{ij}(n)\to 0$ for every $i,j\in C$, but since this class $C$ is closed this gives the contradiction $1=\lim_n \sum_j p_{ij}(n)=0.$