3
$\begingroup$

Consider a CTMC with state space on $\{3,4,5,..\}\times\{3,4,5,...\}$. The exponential rate $\gamma_{(i,j)}$ in state $(i,j)$ equals $i$. $P_{(i,j)\to(3i,j)}=\frac{1}{\sqrt{i}}$ and $P_{(i,j)\to(i,j+1)}=1-\frac{1}{\sqrt{i}}$. Let $T_k$ denote the time of the kth transition in the CTMC. If the CTMC is initialized to state (3,3) at time 0. Compute $\lim_{k\to\infty}E[T_k]$.

In my opinion, it seems as $i$ grows, it is more likely that DTMC goes up. But I have no idea how it relates to the result.

  • 0
    Detailed hint: Compute the mean number of "vertical" jumps when on the line $\{i\}\times\{3,4,5,\ldots\}$ before a jump to the line $\{3i\}\times\{3,4,5,\ldots\}$ occurs. Compute the mean time $v(i)$ all these jumps take. Compute the time $h(i)$ the "horizontal" jump from the line $\{i\}\times\{3,4,5,\ldots\}$ to the line $\{3i\}\times\{3,4,5,\ldots\}$. Then your limit is $$\sum_{n=1}^\infty v(3^n)+h(3^n)$$ which happens to be finite. – 2017-02-26
  • 0
    It seems the CTMC on the line $\{i\}\times\{3,4,5,...\}$ is a sum of i.i.d. exponential distribution with rate $i$, and the sum obeys geometric distribution with success probability $1/\sqrt{i}$. Therefore the sum is exponetial distribution with rate $i\frac{1}{\sqrt{i}}=\sqrt{i}$. Thus its mean is $1/\sqrt{i}$. Therefore the limit should be $\sum_{n=1}^\infty 1/\sqrt{3^n}=\frac{\sqrt{3}+1}{2}$. – 2017-02-26

0 Answers 0