-1
$\begingroup$

Given a simple CTMC with three states 0,1,2. There are three transitions $0 \rightarrow 1$ (with rate $2u$), $1 \rightarrow 2$ (with rate $u$), $1 \rightarrow 0$ (rate $v$).So $2$ is an absorbing state. Can someone please help me understand how to find the expected time to reach state-2 from state-0? Using the understanding I get, I want to do the analysis on some complex markov chains.

  • 0
    Did you try the standard method? That is, introduce the expected time t(x) to reach state 2 starting from state x, find a linear system which the t(x) solve, deduce every t(x), and in particular t(0), which is the expected time you are after.2012-09-02

2 Answers 2

4

Since the OP stays silent, here is an answer. As explained in a comment, the standard method is to introduce the expected time $t_x$ to reach state $2$ starting from each state $x$, to find a linear system which the vector $(t_x)_x$ solve, to deduce $(t_x)_x$, and in particular $t_0$ which is the expected time asked for. Here, the vector is simply $(t_0,t_1)$, and $t_0=\frac1{2u}+t_1,\qquad t_1=\frac{1}{u+v}+\frac{u}{u+v}\,0+\frac{v}{u+v}t_0,$ hence $t_0=\frac1{2u}+\frac1{u+v}+\frac{v}{u+v}t_0,$ which yields finally $t_0=\frac{3u+v}{2u^2}.$

0

You do it in a similar way as you would for finding the expected first passage time for a discrete chain. If $Q = q_{ij}$ is the transition rate matrix, $P = p_{ij}$ is the transition matrix of the embedded chain, \operatorname{Boole} is the indicator function and $T[i, Q]$ is the first passage time starting at state $i$, then

$\begin{align*} \operatorname{\operatorname{Mean}}[FPT[i, Q]] &= \operatorname{Expectation}[FPT \mid X[0] = i]\\ &= \operatorname{Expectation}[\operatorname{SojournTime} \mid X[0] = i] + \sum _{j=1}^{\infty} p_{ij} \operatorname{Expectation}[FPT \mid X[0] = j]\\ &=-\frac1{q_{ii}} + \sum _{j=1}^{\infty} \frac{-q_{ij}}{q_{ii}} \operatorname{Boole}[j\neq i] \operatorname{Mean}[FPT[j, Q]] \end{align*}$

where the sojourn/holding time is, of course, exponentially distributed with parameter $-q_{ii}$.