8
$\begingroup$

Let the following transition matrix represent a $4$ state Markov chain $\begin{pmatrix} 0 & a & 0 & b \\ \frac{1}{2} & 0 & \frac{1}{3}+c & d \\ 0 & a & 0 & b \\ e & 0 & f & 0 \end{pmatrix}$ Let all the constants be positive real numbers, what values of these would make the chain aperiodic? Any help here is greatly appreciated, because I can't see how to do this other than finding an expression for the $n$th power of the matrix at the diagonal values

  • 0
    For this to be the transition matrix of a Markov chain, never mind whether it's aperiodic or not, the rows must sum to $1$.2012-02-22

2 Answers 2

8

If all of $a,b,c,d,e,f$ are $>0$, then the chain is aperiodic. Starting in state 2 you can go $2\to1\to2$ or $2\to4\to1\to2$ with non-zero probability. Hence, the period of state 2 divides both 2 and 3, and so the period of state 2 is 1. Since the chain is irreducible, all states are aperiodic.

If, for example, $a=0$ then the chain is not even irreducible.

  • 0
    Right, I think I understand, is the only condition on the constants the one you stated? I find this concept of aperiodic hard to get my head around2012-02-22
9

More generally: aperiodicity of a Markov chain depends only on the underlying directed graph. The actual values of the nonzero entries of the transition matrix don't matter, as long as they are nonzero.

EDIT: A picture may help. Here is the directed graph with the states $1,2,3,4$ and an arrow from $i$ to $j$ wherever the $(i,j)$ entry in the transition matrix is nonzero.

enter image description here

Consider starting at a state $i$, travelling along the arrows, and ending back at $i$. If every way of doing this has a number of steps divisible by $d$, and $d$ is the largest integer with this property, then state $i$ has period $d$. If $d=1$, the state is aperiodic. As Byron said, starting, say, at state $2$ you can return to $2$ in $2$ steps ($2 \to 1 \to 2$) or $3$ steps ($2 \to 4 \to 1 \to 2$), so state $2$ is aperiodic. On the other hand, if you removed the arrow from $2$ to $4$ (i.e. made $d=0$), all states would have period $2$.