7
$\begingroup$

I have the following homework problem:

Let $(X_n)_{n \geq 0}$ be a Markov chain on the state space $\lbrace0,1,...\rbrace$. Writing $p_i := p_{i,i+1}$ and $q_i := p_{i,i-1}$, the transition probabilities are $p_0 = 1, \qquad \mathrm{and}\quad p_i + q_i =1, \quad \frac{q_i}{p_i} = \left(\frac{i}{i+1}\right)^2 \mathrm{for} \; i \geq 1$

I am asked to show that almost surely we have $X_n \rightarrow \infty$ as $n \rightarrow \infty$.

[Incidentally, I'd like to check my definition of almost sure convergence in this case: I'd say that $X_n \rightarrow \infty$ almost surely just when for every $K\geq 0$ there is $N=N_K$ such that $\mathbb{P}(X_n \geq K \;| \;n \geq N) = 1.] $

It is easy to see that $p_i > 1/2$ for all $i$. The result is then I think intuitively clear by comparison to the asymmetric random walk on the same state space with $p_i = p > 1/2$ for all $i$; the limit in that case is a consequence of the strong law of large numbers.

Now write the hitting probability $h_i^k := \mathbb{P}(X_n = k \textrm{ for some }n \geq 0 \; | \; X_0 = i)$. I believe the result would follow if I could show that

(a) wherever $i \leq k$ we have $h_i^k = 1$, and
(b) for any fixed k, we have $\lim_{i \rightarrow \infty} h_i^k = 0$

Questions: (1) Does the result indeed follow from (a) and (b)?
(2) (b) is clear. How might one show (a)?

  • 0
    Actually, re-reading my first comment I think I may be on to something now: The probability of returning to state $0$ is 1- 6/\pi^2 < 1, so state $0$ is transient. The chain is irreducible, so every state is transient. Though the question is not very clear in this regard, I think we have $X_0 = 0$. With probability 1, we make only finitely many visits to $0$, so we at some stage forever leave the set $\lbrace 0 \rbrace$. Similarly we then a.s. forever leave $\lbrace 0,1 \rbrace$, and next forever leave $\lbrace 0,1,2 \rbrace$, and so forth. Hence $X_n \overset{a.s.}\rightarrow \infty$.2011-11-26

1 Answers 1

3

In the last comment you gave all the necessary ideas for the solution, so let us summarize and formalize them.

Let us denote by $\mathscr X = \{0,1,2,\dots\}$ the state space, the set $A_k = \{0,1,2,\dots,k\}\subset \mathscr X$ and an events $ \mathrm A_k = \{X_n \in A_k\text{ for infinitely many }n\} $ and its complement $ \mathrm B_k = \{\text{there is }N:X_n\in A_k^c \text{ for all }n\geq N\} $ and note that $\mathrm B_{k+1}\subset \mathrm B_k$.

  1. $\mathsf P_i$-a.s. convergence $X_n\to \infty$ means that for any $i\in\mathscr X$ it holds that $\mathsf P_i\{\mathrm B_k\} = 1$ for all $k\geq 0$ or equivalently $ \mathsf P_i\left\{\bigcap\limits_{k=0}^\infty\mathrm B_k\right\} = \lim\limits_{k\to\infty}\mathsf P_i\{\mathrm B_k\} =1. $

  2. You've already noticed that $\mathscr X$ is a closed communicating transitive class. Thus $ \mathsf P_i\{X_n = i\text{ for infinitely many }n\} = 0. $ As a consequence $\mathsf P_i\{\mathrm A_k\} = 0$ since $A_k$ is finite, hence $\mathsf P_i\{\mathrm B_k\} = 1$ and $\lim\limits_{k\to\infty}\mathsf P_i\{\mathrm B_k\} =1$.