0
$\begingroup$

Suppose I have a finite set say $C=\{c_0,c_1,\ldots,c_R\}$ which are some disjoint subsets of the space say $\{0,1\}^m$. The cardinality of the union of these subsets is much less than $2^m$ say.

If I start randomly from any subset of this class $C$ (say draw uniformly according to its size) then I move to the next state according to a matrix $P$ which describes probabilities of moving from any class to any other class inside $C$ or going to $\{0,1\}^m\setminus C$. If the probability of going from $\{0,1\}^m\setminus C$ to $C$ is zero is there any chance of having an invariant distribution for this model?

I find that reccurence and irreducibility of the graph ensure the existence of an invariant distribution. However, is there any chance of having an invariant distribution under this scenario?

  • 0
    What I wa$n$t to see actually is that if there is a cha$n$ce of an invariant distribution if the graph does not have at least one of the properties recurence and irreducibility2012-11-26

2 Answers 2

0

It seems that the situation you have in mind is that the state space $S$ of the Markov chain can be partitioned into a disjoint union $S=S_0\cup S_1$ such that every transition probability $Q(x,y)$ from a state $x$ in $S_1$ to a state $y$ in $S_0$ is zero. Then the restriction of $Q$ to $S_1\times S_1$ is a transition matrix as well hence $Q$ has an invariant distribution $\pi$ which is concentrated on $S_1$, that is, such that $\pi(S_1)=1$ and $\pi(S_0)=0$. If furthermore, for every $x$ in $S_0$ there exists a path to $S_1$ which has positive probability, then every invariant distribution of $Q$ is concentrated on $S_1$.

  • 0
    You switched $S_0$ and $S_1$. With the notations of your comment, $s_0$ is called an absorbing state and the only stationary distribution is the Dirac probability at $s_0$ (as soon as $s_0$ is accessible starting from every other state).2012-11-28
0

In order for an invariant distribution to exist, it is necessary and sufficient that at least one state is positive recurrent. In particular, every Markov chain with a finite number of states has a stationary distribution.

If your chain is not irreducible, just pick a closed irreducible subset. Since your chain is finite, this subset will be a positive recurrent, irreducible chain and hence has an invariant distribution. Now extend this distribution to the rest of the states by giving them measure zero.

If your chain has more than one closed irreducible subset, it will have more than one invariant distribution (indeed, infinitely many).