3
$\begingroup$

Let $S$ be a countable set and $\pi$ a probability distribution on $S$. A discrete-time Markov chain $(X_n)$ with state space $S$ is said to be in detailed balance with respect to $\pi$ (or simply in detailed balance) if for all states $x$ and $y$, $\pi(x)P(x \to y) = \pi(y)P(y \to x).$ (a) Show that if $(X_n)$ satisfies detailed balance, then $\pi$ is a stationary distribution for $(X_n)$.

(b) Consider the general two-state chain with $P(0 \to 1) = p$ and $P(1 \to 0) = q$, where $p,q > 0$. Let $\pi$ be the (unique) stationary distribution. Show the two-state chain always satisfies detailed balance with respect to $\pi$.

(c) Find an irreducible 3-state chain that does not satisfy detailed balance.

(d) Show that any irreducible, positive-recurrent birth-death process satisfies detailed balance with respect to its (unique) stationary distribution.

  • 0
    from $(1)$ we get $\pi(0)(1-P(0 \to 0))=\pi(0)P(0 \to 1)=\pi(1)P(1 \to 0)$ from $(2)$ we get $\pi(1)(1-P(1 \to 1))=\pi(1)P(1 \to 0)=\pi(0)P(0 \to 1)$ and I would say it' done. Or not? Have no ideas for $(c)$ and $(d)$2012-09-20

1 Answers 1

2

Your answers to parts and (a) and (b) appear correct to me, so I will focus on parts (c) and (d).

For part (c), consider the Markov chain with transition matrix,

$ P = \begin{pmatrix} 0 & 1 & 0\\ 0 & 0 & 1\\ 1 & 0 & 0 \end{pmatrix}. $

Then the stationary distribution is given by $\pi=(1/3,1/3,1/3)$, which can be checked by a direct multiplication $\pi P = \pi$. However, the detailed balance equations do not hold.

For part (d), the answer is somewhat heavy in terms of notation, but I will do what I can. Suppose, that you have a birth-death process with birth rates $\{\lambda_i\}_{i\geq0}$ and death rates $\{\mu_i\}_{i\geq1}$. Because the birth-death process is assumed to be positive recurrent, the stationary distribution exists and has the following form.

$ \pi_n=\frac{1}{c} \frac{\prod_{i=0}^{n-1} \lambda_i}{\prod_{i=1}^{n} \mu_i} $

The constant $c$ is given by

$ c=\sum_{n=0}^\infty \frac{\prod_{i=0}^{n-1} \lambda_i}{\prod_{i=1}^{n} \mu_i} < +\infty. $

The summation is finite by the assumption of positive recurrence. Using these expressions, you can check that the detailed balance equations will hold.

I did not include a derivation of the stationary distribution because your question did not ask for it, but I can provide that as well if you want. They are derived by solving the detailed balance equations for $\pi$.