0
$\begingroup$

Again a probability exercise:

Let $X=U \cup V$ be the finite state space of a Markov chain, where $U$ and $V$ are disjoint subsets of $X$ and $p_{ij}=0$ if both $i,j \in U$ or both $i,j \in V$. Otherwise, they are positive. Find a stationary distribution?

I'm assuming that it has multiple distributions but I wrote down the system that I'm trying to solve and either I'm really tired or I should be thinking of smth else: if $\pi$ is the stationary distribution then I'm trying to find we have $\pi_{j} = \sum_{i \in V} {\pi_{i} p_{ij}}$ for all $j \in U$ and also $\pi_{j} = \sum_{i \in U} {\pi_{i} p_{ij}}$ for all $j \in V$... Can someone please help again?

Thanks!

  • 0
    Anna: Why do you abstain from accepting the answer below? I doubt anything meaningful can be added to it...2012-04-12

1 Answers 1

1

The wording of this exercise seems a bit strange to me, since every finite state space Markov chain has a stationary distribution.


Added: Let $P$ be the transition matrix of your chain. Any stationary distribution $\pi$ for $P$ is also stationary for $P^2$. But under $P^2$, the sets $U$ and $V$ are disjoint, recurrent communicating classes with unique stationary distributions $\pi_U$ and $\pi_V$ (of lengths $|U|$ and $|V|$).

Let $\tilde\pi_U$ and $\tilde\pi_V$ be vectors of length $|X|$ obtained by padding $\pi_U$ and $\pi_V$ with zeros in the appropriate places. Then the set of all stationary distributions for $P^2$ is $\{\alpha\tilde\pi_U+(1-\alpha)\tilde\pi_V,\, 0\leq \alpha\leq 1\}.$

The only one of these that is also stationary for $P$ is ${1\over 2}\tilde\pi_U+ {1\over 2}\tilde\pi_V$.

  • 0
    @Anna Maybe you should add the definition of "stationary distribution" to your question. Usually it means a row vector $\pi$ of probabilities that satisfies $\pi=\pi P$.2012-03-12