5
$\begingroup$

I'm reading through my textbook, Introduction to Stochastic Processes (Lawler), before the semester begins in hopes of getting ahead, and I've run into something I just plain cannot figure out: How to compute the invariant probability vector for a transition matrix. I was hoping that one (or many) of you would be able to walk me through how you would do this for just a simple matrix:

$\begin{bmatrix} .4&.2&.4 \\\\ .6&0&.4 \\\\ .2&.5&.3 \end{bmatrix}$

I know that you can compute it by raising the matrix to a large power, but this practice problem says to "compute the invariant probability vector as a left eigenvector." How would one go about doing this?

Thanks for your help!

  • 0
    Invariant: matrix operation on $\mu$ still gives $\mu$.2012-08-12

3 Answers 3

9

If the transition matrix is $A$ and the probability vector is $\mu$, "invariant" means that $\mu A = \mu$. Another way of saying this is that $\mu$ is a left eigenvector of $A$ with eigenvalue 1.

$\mu A = \mu$ is really just a system of linear equations. If we write $\mu = [\mu_1, \mu_2, \mu_3]$ then we have $[\mu_1, \mu_2, \mu_3] \begin{bmatrix} .4&.2&.4 \\\\ .6&0&.4 \\\\ .2&.5&.3 \end{bmatrix}= [\mu_1, \mu_2, \mu_3]$ or in other words $\begin{align*} .4 \mu_1 + .6 \mu_2 + .2 \mu_3 &= \mu_1 \\ .2 \mu_1 + 0 \mu_2 + .5 \mu_3 &= \mu_2 \\ .4 \mu_1 + .4 \mu_2 + .3 \mu_3 &= \mu_3. \end{align*} $ Since $\mu$ is to be a probability vector we also have to have $\mu_1 + \mu_2 + \mu_3 = 1.$ So you have a system of 4 linear equations in 3 unknowns. Now you just have to solve this system.

  • 2
    Note that the first three equations are linearly dependent since the sum of each row of the matrix is $1$, so you can take any two of those three plus the fourth equation, and solve $3$ equations in $3$ unknowns.2012-08-13
1

It means, find a vector $v$ with the following properties: all entries between 0 and 1, entries add up to 1, and $vA=v$ (where $A$ is your transition matrix).

If you know how to find (left) eigenvectors, you just find one for the eigenvalue 1, and then normalize it so the sum of the entries is 1.

  • 0
    Yep, that's exactly what I know how to do. You've been very helpful; thanks again!2012-08-13
-1

If T is the transistion probability matrix the stationary (or invariant) distribution satisfies the matrix equation

P(X)= T P(X)

  • 1
    My experience is that mathematicians generally set up their stochastic processes with columns summing to 1 and $Av=v$ (like Michael), whereas statistics/economics people have rows summing to 1 and $vA=v$ (like OP). Causes some confusion when the two try to talk to each other.2012-08-13