5
$\begingroup$

Can anyone give me a simple example of an irreductible (all elements communicate) and transient markov chain? I can't think of any such chain, yet it exists (but has to have an infinite number of elements) thanks

  • 0
    But why is it transient? I would need to compute the probability of going back to origin but I don't know how2012-11-21

1 Answers 1

7

A standard example is asymmetric random walk on the integers: consider a Markov chain with state space $\mathbb{Z}$ and transition probability $p(x,x+1)=3/4$, $p(x,x-1)=1/4$. There are a number of ways to see this is transient; one is to note that it can be realized as $X_n = X_0 + \xi_1 + \dots + \xi_n$ where the $\xi_i$ are iid biased coin flips; then the strong law of large numbers says that $X_n/n \to E[\xi_i] = 1/2$ almost surely, so that in particular $X_n \to +\infty$ almost surely. This means it cannot revisit any state infinitely often.

Another example is simple random walk on $\mathbb{Z}^d$ for $d \ge 3$. Proving this is transient is a little more complicated but it should be found in most graduate probability texts.