1
$\begingroup$

$P=\left(\matrix{0.7 \ \ \ \ 0.3\\0.6 \ \ \ \ 0.4}\right)$ be the transition probability matrix of finite markov chain wtih 2 states, $1,2$ and what is the probability that in long run, $P(X_n=1)$?

I know how to find stationary distribution but the solution suggest that it is equal to $\frac{0.6}{0.6+0.3}$ without using any stationary distribution and i don't quite get how it gets that.

  • 2
    Now that's rather original -- usually we have the problem that the title basically says nothing more than "problem" or "question" -- you've written a title in which the only nouns are "solution" and "answer" :-) Please choose a more specific title that summarizes the content of the question.2012-12-16

2 Answers 2

2

You could try to show the following:

The stationary distribution of the Markov chain with transition matrix $\begin{pmatrix}1-a & a\\ b & 1-b\end{pmatrix}$ is proportional to $\begin{pmatrix}b\\ a\end{pmatrix}$, for every $a$ and $b$ in $(0,1)$.

Of course, such a characterization is specific to the two-states case.

  • 0
    I see. As from the computation of the stationary distribution, we can get that $a\pi_1 - b\pi_2=0$ so we have $\frac{\pi_1}{\pi_2}=\frac{b}{a}$.2012-12-16
1

The markov chain is irreducible and finite state. Additionally it is aperiodic. Thus not only is it positive recurrent but it has a unique steady state distribution.

  • 0
    I just wanted to fill in the gaps. He already knows how to solve it.2012-12-16