2
$\begingroup$

Given a markov process $\mathcal{Y} = (Y_t : t \ge 0)$ with state space $E=\{1,2,3\}$ and with generator matrix

$G = \left[ \begin{array}{ccc} -3 & 1 & 2 \\ 1 & -2 & 1 \\ 0 & 0 & 0 \end{array} \right]$.

Assume that the initial distribution is given by $\alpha_1 = \mathbb{P}(Y_0 =1) = 3/4$ and $\alpha_2 = \mathbb{P}(Y_0 =2) = 1/4$.

How do I find the distribution of the first hitting time $Z: = \min\{t \ge 0 : Y_t =3\}$ of state $3$?

How would I also write down the distribution function $F(x) = \mathbb{P}(Z\le x)$ and the density function $f(x)$ on $x>0$?

1 Answers 1

1

Let $Z_i$ denote the first hitting time of $3$ when $Y_0=i$. Then the distribution of $Z$ is the barycenter of the distributions of $Z_1$ and $Z_2$ with weights $\alpha_1$ and $\alpha_2$.

Furthermore, $Z_1=\frac13T+BZ'_2$ where $(T,B,Z'_2)$ are independent, $T$ is standard exponential, $B$ is Bernoulli with probability $\frac13$ of success, and $Z'_2$ is distributed like $Z_2$.

Likewise, $Z_2=\frac12T'+B'Z'_1$ where $(T',B',Z'_1)$ are independent, $T'$ is standard exponential, $B'$ is Bernoulli with probability $\frac12$ of success, and $Z'_1$ is distributed like $Z_1$.

Let $u_i(s)=\mathrm E(\mathrm e^{-sZ_i})$ and $t(s)=\mathrm E(\mathrm e^{-sT})=(s+1)^{-1}$. Then, $ u_1(s)=t(\tfrac13s)\cdot(\tfrac13u_2(s)+\tfrac23),\qquad u_2(s)=t(\tfrac12s)\cdot(\tfrac12u_1(s)+\tfrac12), $ that is, $ u_1(s)=\frac{2+u_2(s)}{3+s},\qquad u_2(s)=\frac{1+u_1(s)}{2+s}, $ which yields $ u_1(s)=\frac{5+2s}{5+5s+s^2},\qquad u_2(s)=\frac{5+s}{5+5s+s^2}. $ Equivalently, $ u_1(s)=\frac{a}{1+as}+\frac{1-a}{1+(1-a)s},\qquad u_2(s)=\frac{b}{1+as}+\frac{1-b}{1+(1-a)s}, $ for $a=\frac12(1+\frac1{\sqrt5})$ and some $b$ in $(0,1)$. Finally, $ \mathrm E(\mathrm e^{-sZ})=\alpha_1u_1(s)+\alpha_2u_2(s)=\frac{A}{1+as}+\frac{1-A}{1+(1-a)s}, $ with $A=\alpha_1a+\alpha_2b$, hence the density of the distribution of $Z$ is $ f(z)=\left(\frac{A}a\mathrm e^{-z/a}+\frac{1-A}{1-a}\mathrm e^{-z/(1-a)}\right)\cdot[z\geqslant0]. $

  • 0
    Thanks @Didier! Have been looking through what you did slowly, in order to understand it. Most appreciated again. Cheers :)2012-04-18