0
$\begingroup$

This is not a wikipedia question, because I cannot find it there. How does one compute the first return time of a time-homogeneous Markov chain. There are a lot of neat notions such as recurrence and expected return time, but I am not sure how one can calculate the first return time.

For simple example, if we have $p_{11} =1/3$, $p_{12}=2/3$, $p_{22}=1/6$ and $p_{21}=5/6$, then what is the first return time to state 1?

  • 0
    It looks like you have "$p_11$" where you want $p_{11}$. If so, write "p_{11} instead of "p_11".2012-01-26
  • 0
    @Brian You have two different values for $p_{12}$.2012-01-26
  • 0
    @Bryon / Brian: corrected so $p_{22}+p_{21}=1$2012-01-26
  • 0
    The first return time is a random variable. In what sense do you want to "calculate" it?2012-01-26
  • 0
    expectation of the random variable i mean2012-01-26
  • 0
    Then what do you mean by "expectation of the first return time" that makes it different from "expected return time"?2012-01-26
  • 0
    no it doesnt actually ok2012-01-26

3 Answers 3