3
$\begingroup$

For a discrete time Markov chain with state-space the non-negative integers, for $j>0$,

$ p_{j,k} = \begin{cases} p/j & \text{for } k = j+1 \\ 1 - 1/j & \text{for }k=j \\ (1-p)/j & \text{for }k=j-1 \end{cases} $ and $p_{0,1}=p,\ p_{0,0}=1-p$.

Find the ranges of $p$ such that the process is transient, null-recurrent, and positive-recurrent.

Thanks in advance!

Progress: For showing that it is transient, initially I tried summing $p(n)_{0,0}$ (the probability of returning to state 0 in n steps) from n=1 to infinity and finding the range of p for which this sum converged, which would suggest transience, and diverged, which would suggest recurrence. This however involved a double summation of factorials which was too hard to solve.

Then I tried finding the general solution to $f_{j,0}$, the probability that, starting in j, the process will ever reach state 0. I became stuck at a messy recursion equation: $jf_{j,0}=pf_{j+1,0}+(1-p)f_{j-1,0}+j-1$. Finding p for which $f_{0,0}=1$ would imply recurrence.

Then I tried solving the stationary distribution, pi=pi*P, but this too resulted in a nasty recursion equation: $pi_{j}=p*j/(j-1)*pi_{j-1}+(1-p)*j/(j+1)*pi_{j+1}$. My lecturer says that there is a simplification which can make these recursion equations solvable but I am yet to find see it...

Update: I think I've got it.

  • 0
    Thanks for the pointers! I'm kind of new to this site so an$y$ tips are welcome! Also, I have a $f$eeling that the 'one line answer' is transience: p>0.5, positive-recurrence: p<0.5 and null-recurrence: p=0.5. I will need to show the 'process' o$f$ getting there though.2011-09-03

0 Answers 0