1
$\begingroup$

Why we can calculate the limiting probability of $\begin{pmatrix}0.6&0.2&0.1&0.1\\0.6&0&0.3&0.1\\0&0.6&0&0.4\\0&0&0.6&0.4\end{pmatrix}$ in this way?

Solution: Taking $\begin{pmatrix}\pi_0&\pi_1&\pi_2&\pi_3\end{pmatrix}(I-P)=$$\begin{pmatrix}\pi_0&\pi_1&\pi_2&\pi_3\end{pmatrix}\begin{pmatrix}-0.4&0.2&0.1&0.1\\0.6&-1&0.3&0.1\\0&0.6&-1&0.4\\0&0&0.6&-0.6\end{pmatrix}=0$ and we get the limiting probability

I don't quite get why it can be calculated in this way as state 2 is obviously not aperiodic so can this method being used? If not, how should one calculate it?

  • 0
    One of you could write that as an answer.2012-12-18

1 Answers 1

2

Call your matrix $M$. Note that both $M^3$ and $M^4$ have all strictly positive entries. This means there is a positive probability that from state $k=1,2,3,4$ that there will be return to state $k$ in three steps, and in four steps. Any integer $t$ greater than $3 \cdot 4-3-4=5$ has an expression $t=3u+4v$ where $u,v \ge 0$. It follows that for $N=6$ we can say that, for any $n \ge N$, the recurrence probability from state $k$ to state $k$ in $n$ steps has positive probability. This shows all four states $1,2,3,4$ are aperiodic, since aperiodic only means positive recurrence probability at $n$ steps, for all sufficiently large values of $n$.

NOTE: For this matrix $M$ in fact all one needs is that each of $M^2$ and $M^3$ has all strictly positive entries on the diagonal. Then for all $n \ge 2$ one can as above show there is positive probability, starting at state $k$, of returning to state $k$ in $n$ steps.

  • 0
    "One positive entry on the diagonal + irreducibility" guarantees aperiodicity.2012-12-21