0
$\begingroup$

Is it that the values in each column add up to one? Also, how does this work?

3 Answers 3

4

The column sums of the transition matrix are not relevant to your problem.

Here is Theorem (3) from section 6.4 of Probability and Random Processes (3rd edition) by Grimmett and Stirzaker.

An irreducible chain has a stationary distribution $\pi$ if and only if all the states are non-null persistent; in this case, $\pi$ is the unique stationary distribution and is given by $\pi_i=\mu_i^{-1}$ for each $i\in S$, where $\mu_i$ is the mean recurrence time of $i$.

Their term "non-null persistent" is often called "positive recurrent" by other authors. This stationary distribution $\pi$ will also be the limiting distribution (i.e. $X_n\to \pi$ in distribution) provided that the chain is, in addition, aperiodic.


Added: On re-reading this, it looks like I've sidestepped the question but substituting one abstract probabilistic condition for another. Let me try to answer in the OP's language.

I will assume a finite state space.

What conditions on the transition matrix $P$ guarantee the existence of a limiting distribution? That is, $\pi=\lim_n {\cal L}(X_n)$ where $\cal L$ means "law" and the limit is "in distribution". Since we do not specify the initial state, we will need that every recurrent state is aperiodic.

In linear algebra terms, this is true if and only if every eigenvalue (except 1) of $P$ has modulus strictly less than 1.

These references may be helpful: Perron–Frobenius theorem, Markov Chains in discrete time.

  • 0
    @Tim I only say that $\lim_n P^n$ exists, not that it has identical rows. The rows will be identical when the eigenvalue 1 has multiplicity one.2012-05-02
0

I've not done Markov processes before but if you look here it seems that

(i) the rows sum to 1 because otherwise the transition probabilities are not a probability distribution

(ii) to compute the stationary distribution you have to compute $\mathbb{\pi}$ as left eigenvector of the transition matrix $P$: $\mathbb{\pi} P = \mathbb{\pi}$

Does that make any sense? If it does then the answer to your question would be that the stationary distribution exists if $P$ has a left eigenvector.

  • 0
    @Byron: thanks Byron, I corrected it.2011-03-22
0

In short, every irreducible + aperiodic transition matrix (primitive matrix as Wikipedia) will have a stationary distribution vector. These Wikipedia links will help you to have a good perspective in Matrix algebra terms: https://en.wikipedia.org/wiki/Stochastic_matrix https://en.wikipedia.org/wiki/Perron%E2%80%93Frobenius_theorem

For a statistical interpretation of these results, I recommend this very short read: Finite Markov Chains and Algorithmic Applications - Olle Haggstrom. London Math Society. Student Texts 52. Chapter 4 and 5.

Best regards! Eric.