3
$\begingroup$

I am asked to compute the stationary distribution of the markov chain with state space $E=\mathbb{N}_0$, $q_n >0$ for all $n \in \mathbb{N}_0$ and transition matrix below:

\begin{bmatrix} q_0 & q_1&q_2&q_3 &q_4&q_5&\dots \\ 1 & 0 & \\ & 1 & 0 & & \\ & & 1 & 0 & \\ & & & 1 & 0 & \\ & & & & 1 & 0 \\ & & & & & \ddots &\ddots \\ \end{bmatrix}

I used $\pi P = \pi$ and I also tried the way from my other thread Calculating stationary distribution of markov chain. Letting $g(x) = \sum_{k=0}^n x^k \pi_k$ What I got was:

$\pi_0 = \pi_1 +q_0\pi_0 \\ \pi_1 = \pi_2 +q_1\pi_0 \\ \pi_2 = \pi_3 +q_2\pi_0 \\ \pi_k =\pi_{k+1}+q_k\pi_0 \\ x^k\pi_k =x^k\pi_{k+1}+x^kq_k\pi_0 \\ \sum_{k=0}^{n}x^k\pi_k = \sum_{k=0}^{n}x^{k-1}\pi_{k}+\sum_{k=0}^{n}x^kq_k\pi_0 \\ g(x) =x^{-1}g(x)+\sum_{k=0}^{n}x^kq_k\pi_0$

and also

$\pi_0=\frac{1}{1-q_0}\pi_1 \\ \pi_1 =\pi_0(1-q_0) \\ \pi_2 =\pi_0(1-q_0-q_1) \\ \pi_3 =\pi_0(1-q_0-q_1-q_2) \\\pi_n = \pi_0(1-\sum_{k=0}^{n-1}q_k) \\$

I tried fiddling with it here and there but I cant seem to get anywhere to finish this problem. i.e. I can't seem to find $\pi_k$ for all $k \in E=\{0,\dots,n\}$. How would I finish this problem?

Edit: $\pi_n = \pi_0(1-\sum_{k=0}^{n-1}q_k) \\ \sum_{n=0}^{\infty}\pi_n =1 \\ \sum_{n=0}^{\infty}\pi_0(1-\sum_{k=0}^{n-1}q_k) =1 \\ \pi_0 = \frac{1}{\sum_{n=0}^{\infty}(1-\sum_{k=0}^{n-1}q_k)} = \frac{1}{(1-q_0)+(1-q_o-q_1)+(1-q_0-q_1-q_2)+\dots}$

Looking at @BrianMScott 's comment, how do I get the bottom line to simplify to $\sum_{k\ge 0}(k+1)q_k$? /To simplify the last line above?

  • 1
    You're almost there, you just have to find $\pi_0$. And there's one fact you haven't used yet: you want $\pi$ to be a probability distribution, so you need $\sum_{k=0}^\infty \pi_k = 1$...2012-05-04
  • 0
    Thanks @NateEldredge , I've used your hint and continued on, could you possibly take a look?2012-05-04
  • 2
    Since $\sum_{k=0}^\infty q_k = 1$, we have $\sum_{n=0}^\infty (1-\sum_{k=0}^{n-1} q_k) = \sum_{n=0}^\infty \sum_{k=n}^\infty q_k$. Now try interchanging the order of summation.2012-05-04
  • 1
    Note that your written-out denominator starts at $n=1$; it should start at $n=0$ with a term $1$; this is where the $+1$ in Brian's $k+1$ comes from.2012-05-04
  • 0
    Thanks @NateEldredge , I understood what you did and I got the correct answer. @ joriki Thanks for pointing it out, I did I start with $n=0$, but since it turns out to be $(1-1) =0$, I did not write it out in my workings above.2012-05-04
  • 1
    @Richard: Where does the $-1$ come from? As I wrote above, the $n=0$ term is $1$; the sum over $q_k$ from $k=0$ to $-1$ is $0$. By the way, pinging doesn't work if you leave a space between the @ and the user name; I read this comment by chance.2012-05-04
  • 0
    @joriki Sorry, I got mixed up with sums and products (empty products are $1$). May I ask, did you have 'another' way to get $\sum_{k\ge 0}(k+1)q_k$? Now the denominator reads $1+(1-q_0)+(1-q_o-q_1)+(1-q_0-q_1-q_2)+\dots$, is there another way to get to $\sum_{k\ge 0}(k+1)q_k$ apart from using Nate's method in the above comment?2012-05-04
  • 1
    @Richard: No, that's how I'd do it.2012-05-04

1 Answers 1