0
$\begingroup$

Let be a probability distribution on the nonnegative integers such that $\pi_i > 0$ for all i. Write down the transition matrix of an irreducible, aperiodic, recurrent Markov chain on the nonnegative integers that has as its stationary probability distribution.

File can be found here: https://tbp.berkeley.edu/examfiles/stats/stats150-sp07-final-Evans-exam.pdf (#1 on this site)

What I notice is that I am trying to reduce it to a finite case, then go from specific to general. Maybe I can try the geometric distribution but I not really seeing how using the detailed balance equations here. Many Thanks

  • 1
    Since he requires the chain to be aperiodic, he must mean a discrete time chain, not a continuous time chain.2012-05-05
  • 0
    Can you please draw the transition matrix? Thanks2012-05-05
  • 0
    $P$ is an infinite matrix with identical rows. All of the rows look like $(\pi_1,\pi_2 ,\pi_3,\dots)$.2012-05-05

1 Answers 1

2

The easiest example is to let $P_{ij}=\pi_j$ for all non-negative integers $i,j$. This corresponds to a chain that jumps to state $j$ with probability $\pi_j$ regardless of the initial state. Starting at time 1, the Markov chain $(X_n)_{n\geq 1}$ consists of i.i.d. random variables.

Since $P_{ij}>0$ and $P_{ji}>0$, all states communicate so the chain is irreducible.

Since $P_{ii}>0$ the period of state $i$ is 1, so the chain is aperiodic.

Since $\pi$ is an invariant probability vector, the chain is positive recurrent.