0
$\begingroup$

For given number $p \in (0,1)$ consider Markov chain of states $E = \mathbb{Z}$ and transition matrix such that $p_{0,1} = p_{0,-1} = \frac{1}{2}$ and $p_{k,k+1} = p_{-k, -k-1} = p ,$ $ \ \ p_{k,k-1} = p_{-k -k+1} = 1-p$ for $k=1,2,...$. Check recurrence of this Markov chain.

We know for irreducible Markov chain every state is either recurrent or transient so it's enough to examine recurrence of state $0$. We have criterium $\displaystyle \sum _{n=1}^{\infty}p_{00}(n) = \infty$ then state $0$ is recurrent but here it's seems hard to calculate $p_{00}(n)$ and I have no idea how to find it. Another approach would be checking if $\displaystyle \sum _{n=1}^{\infty}f_{00}(n) = 1$ where $f_{00}(n)$ is probability of first visit in $0$ leaving from $0$ but again i don't know how to finish it.

  • 0
    The probability $u_1$ to hit $0$ starting from $\pm1$ solves $u_1=pu_1^2+1-p$ hence $u_1=1$ or $u_1=(1-p)//p$. If $p\leqslant1/2$, $(1-p)/p\geqslant1$. Since $u_1\leqslant1$, this yields $u_1=1$, as desired. Note that if $p>1/2$, then the chain is transient. – 2017-01-30
  • 0
    how to justify $u_1= pu_1^2+1-p$? with prob. $1-p$ we go from $\pm 1$ to $0$ so we are done but with prob $p$ we go to $\pm 2$ and why we hit 0 then with prob. $u_1^2$ ? – 2017-01-30
  • 0
    Because, to hit $0$ starting from $\pm2$ is to first hit $\pm1$ and then, starting from $\pm1$, to hit $0$. Each of these events has probability $u_1$ and the Markov property allows to conclude. – 2017-01-30

0 Answers 0