1
$\begingroup$

By "isolated", I mean that each state of this Markov Chain has 0 probability to move to another state, i.e. transition probability $p_{ij} = 0$ for $ i \ne j$. Thus, there isn't a unique stationary distribution.

But by definition, since for any stationary distribution $\pi$, we have

$ \pi_{i}p_{ij} = 0 = \pi_{j}p_{ji} $

seems that we can still call this Markov Chain time reversible.

Is the concept "time reversible" still make sense in this situation?


A bit background, I was asked to find a Markov Chain, with certain restrictions, that is NOT time reversible. But I found if the stationary distribution exist, the chain is always reversible. So I guess that my be chance is that a chain who doesn't have unique stationary distribution. Maybe in this situation we can't call the chain reversible.

  • 0
    That depends on the precise definition of Markov chain. As far as I can see, it's still one chain, just one with special transition probability.2011-11-21

2 Answers 2

1

This Markov chain is reversible --- you've shown it yourself by showing that the detailed balance equation holds.

You can show that the detailed balance condition is equivalent to:

  1. $p_{ij}>0\Rightarrow p_{ji}>0$
  2. For every cycle $i_0,i_1,\dots,i_n,i_0$ in the set of states, the probability of 'going' in the cycle $i_0\rightarrow i_0$ is equal for clockwise and anticlockwise orientations.

Thus, you could say that your Markov chain is vacuously reversible as there are no cycles.

0

I do not think that constructing a markov chain with isolated states will give you a time irreversible markov chain.

Consider the case when you have one isolated state. Since, an isolated state can never be reached from any other state, your chain is actually a union of two different markov chains.

  1. A markov chain that always stays in the isolated state (which is time reversible by definition) and

  2. A markov chain on the non-isolated states which may or may not be time reversible.

Thus, I do not think the above strategy will work.