1
$\begingroup$
  1. In a continuous-time Markov chain, I was wondering why the holding time and the next state are independent? Are the independence a conditional one given the current state?

  2. Quoted from Ross's Stochastic processes:

    The amount of time the process spends in state $i$, and the next state visited, must be independent random variables. For if the next state visited were dependent on $\tau_i$, then information as to how long the process has already been in state $i$ would be relevant to the prediction of the next state—and this would contradict the Markovian assumption.

    One can also find identical claim at another book here with more
    context available.

    I don't understand why if the two are dependent, the Markov property is violated.

Thanks and regards!

2 Answers 2

1

For question 1, the answer is yes. You might first consider the case of 2 possible "next states", which corresponds to showing that if $U,V$ are two independent exponential random variables (with possibly different rates), then $\min(U,V)$ is independent of the event $U < V$. Counterintuitive but true.

For question 2, consider a simple example: suppose $X_t$ is a process that, whenever it enters a certain state $a$, it does one of two things: waits 5 seconds and then transitions to $b$, or waits 10 seconds and then transitions to $c$. Here the holding time is completely correlated with the next state. Such a process cannot be Markov. For instance, we have $P(X_{20} = b | X_{17} = a) > 0$, but $P(X_{20} = b | X_{17} = a, X_{14} \ne a) = 0$.

Essentially, information about the holding time is information about the history of the process (where it was at an earlier time), and in a Markov process this is not allowed to prejudice where it goes next.

  • 0
    Thanks! (1) I was wondering why given the current state, it corresponds to **independent** U and V? (2) When U and V are independent exponential r.v.s, why min(U,V) is independent of the event UP(X_{20} = b | X_{17} = a) and $P(X_{20} = b | X_{17} = a)$, the intervals are 3. How do you know the P(X_{20} = b | X_{17} = a) > 0 and P(X_{20} = b | X_{17} = a) > 0? – 2011-05-09
1

The next state depends on the recent states, not how long you stayed in them. Notice that this is implicit in the state transition matrix.

  • 0
    There is no information in the state transition matrix about holding times. It merely tells you the distribution of the next state given the past states.2011-05-02