0
$\begingroup$

Consider a Markov chain with state space $\{0,1,2,3,4,5\}$ and transition probability matrix $$P= \begin{pmatrix} 0&1&0&0&0&0\\ 1/5&0&4/5&0&0&0\\ 0&2/5&0&3/5&0&0\\ 0&0&3/5&0&2/5&0\\ 0&0&0&4/5&0&1/5\\ 0&0&0&0&1&0 \end{pmatrix} $$ One can check directly that $P$ has a stationary distribution $\pi$ with $$ \pi=\bigg(\frac{1}{32},\frac{5}{32},\frac{10}{32},\frac{10}{32},\frac{5}{32},\frac{1}{32}\bigg). $$ A theorem says that the stationary distribution for a regular matrix is also its limiting distribution. Unfortunately, $P$ is not a regular matrix (due to the zero diagonal), and thus the theorem is not applicable. But I don't know if the "regular" condition could be relaxed or not. Here is my question:

Is $\pi$ also the limiting distribution of $P$?

1 Answers 1

1

The usual theorem for convergence to the stationary distribution for a finite state space uses irreducibility and aperiodicity. Irreducibility in your case is clear: the accessibility graph is just the undirected straight line graph which is strongly connected. However this chain is actually periodic. For example, all paths from state 1 to state 1 have a length which is a multiple of 2. This is reflected in an eigenvalue of -1 in the transition probability matrix. Accordingly you have convergence to the stationary distribution if and only if your initial distribution is orthogonal to the right eigenvector with this eigenvalue, which turns out to be $\begin{bmatrix} 1 \\ -1 \\ 1 \\ -1 \\ 1 \\ -1 \end{bmatrix}$. So if $p_1+p_3+p_5=p_2+p_4+p_6$ then you have convergence to the stationary distribution, otherwise you don't.

Note that the theorem you stated is a special case of this one, because having all nonzero diagonal entries gives aperiodicity for free.

Also note that you do have Cesaro convergence to the stationary distribution, i.e. $\lim_{n \to \infty} \frac{1}{n} \sum_{k=1}^n p^{(k)} = \pi$.

  • 0
    I would be interested in seeing more details, thanks. For "convergence to the stationary distribution", do you mean for instance $\lim_{n\to\infty}P(X_n=j\mid X_0=i)=\pi_j$? Also, do you have a reference for the statement "convergence to the stationary distribution if and only if your initial distribution is orthogonal to the right eigenvector with this eigenvalue"? I'm a little confused that why the limiting distribution would have anything to do with the initial distribution. – 2017-02-25
  • 0
    @Jack The chain converges to the stationary distribution if its initial distribution is $p$ and $\lim_{n \to \infty} p P^n = \pi$. The orthogonality issue is just saying that if we diagonalize $P=V \Lambda V^{-1}$, then $(p V)_i$ must be zero whenever $|\lambda_i|=1$ and $\lambda_i \neq 1$. Otherwise that component of $p P^n$ will oscillate forever. – 2017-02-25
  • 0
    @Jack Intuitively what's going on in your example is that if the initial state is, say, odd, then the state will be odd at all even times and even at all odd times. Thus the distribution cannot possibly converge for this initial distribution. It instead will converge if the probability of being even at the beginning is the same as the probability of being odd at the beginning. As for dependence of the limiting distribution on the initial distribution, that can happen even when the limiting distribution exists. – 2017-02-25
  • 0
    @Jack (Cont.) This is what happens when the stationary distribution is not unique, which in turn can happen when your chain has two or more weakly connected components. Then each weakly connected component, say $C_i$, has its own stationary distribution, say $\pi^{(i)}$. Then the limiting distribution will be $\sum_i \pi^{(i)} P(X_0 \in C_i)$: intuitively, whatever probability was assigned to $C_i$ initially gets dispersed according to the stationary distribution of $C_i$. – 2017-02-25
  • 0
    Thank you for all your elaboration! I'm slowing reading it. I will be back once I figure out all the details. (Still, do you have a reference for the comments you made? ) – 2017-02-25
  • 0
    @Jack I do not, unfortunately, I learned most of my probability skills on an informal basis in classes and in direct conversation with professors and students. – 2017-02-26