Property of expected hitting time in Markov chains

Question

Giving the probability space with states $i\in \Omega$, the conditional probability of starting at $i$ is $\mathbb{P}_i = \mathbb{P}(.|X_0=i)$.

Giving $A\in \Omega$ , let's define the hitting time : $H^{A} : \Omega \rightarrow \mathbb{N}$ : $$H^{A}(\omega) = \{\text{inf } n, X_n(\omega) \in A\}$$

I've seen many texts mentioned (also in MSE) $$\mathbb{E}_i[H^A|X_1=j] = 1 + \mathbb{E}_j[H^A]$$ by simply saying it is the Markov property. The Markov property as I understood is by shifting 1 the Markov chain, we again have the same Markov chain. So I don't understand where the 1 come from.

Though I understand the intuition, that is the expected time to hit $A$ knowing $\{X_1=j,X_0=i\}$ is 1 plus the expected time to hit $A$ (re-)starting at $\{X_0=j\}$

Can someone help please, with a rigorous mathematical proof.

Answer 1

For any state $i\notin A$, note that $E_i[H^A]=\sum_{n=1}^\infty nP_i(H^A=n)=\sum_{n=1}^\infty n\sum_{j\notin A}P_i(H^A=n, X_1=j)+\sum_{n=1}^\infty n\sum_{j\in A}P_i(H^A=n,X_1=j)$

Now observe if $j\in A$, then $P_i(H^A=n,X_1=j)=0$ if $n>1$ and $=P_i(H^A=1,X_1=j)=P_i(X_1=j)=p(i,j)$ for $n=1$.

Hence $\sum_{n=1}^\infty n\sum_{j\in A}P_i(H^A=n,X_1=j)=\sum_{j\in A}P_i(X_1=j)=P_i(X_1\in A)$.

Further if $j\notin A$ then by Markov property, $P_i(H^A=n,X_1=j)=P_i(X_1=j)P_j(H^A=n-1)=p(i,j)P_j(H^A=n-1)$

Writing all these into $E_i(H^A)$ we get $E_i(H^A)=P_i(X_1\in A)+\sum_{n=1}^\infty n\sum_{j\notin A}p(i,j)P_j(H^A=n-1)=P_i(X_1\in A)+\sum_{j\notin A}\sum_{n=1}^\infty (n+1)P_j(H^A=n)=P_i(X_1\in A)+\sum_{j\notin A}p(i,j)E_j(H^A)+\sum_{j\notin A}p(i,j)=1+\sum_{j\notin A}p(i,j)E_j(H^A) $

So this is the general expression. Try to get your statement by imitaing the above arguments.

Answer 2

The addition of 1 is taking into account that it takes time $1$ to go from $i$ to $j$ in the first place. Without this term, the expected time would just be zero whenever $A$ is reachable from $i$.