Inequality with a homogeneous markov chain and hitting probabilities

Question

I need help with the following exercise:

Let $(X_n)_{n \ge 0}$ be a homogeneous Markov chain with a countable state space $(E,\mathcal B)$. We call $T_A(\omega):=\text{inf}\{ n \in \mathbb N_0: X_n(\omega) \in A \}$ the hitting time with the special case of inf($\emptyset$)=$\infty$.

Now for $x,y,z \in E$ and $f_{xy}:=P_x(T_y < \infty)=P(T_y < \infty \mid X_0=x)$, I have to show that the inequality

$f_{xz} \ge f_{xy} \cdot f_{yz}$

is true.

As a hint I am supposed to use the strong markov property. Unfortunately, I don't know how to show it.

I hope someone can help me here.

Answer 1

$ \begin{align*} P(T_z<\infty\mid X_0=x)&=P(T_z<\infty\mid T_y<\infty, X_0=x)P(T_y<\infty\mid X_0=x) \\ &\quad+P(T_z<\infty\mid T_y=\infty, X_0=x)P(T_y=\infty\mid X_0=x) \\ &\geq P(T_z<\infty\mid T_y<\infty, X_0=x)P(T_y<\infty\mid X_0=x) \\ &=\bigg[P(T_z<\infty\mid T_z\leq T_y, T_y<\infty, X_0=x)P(T_z\leq T_y\mid T_y<\infty, X_0=x) \\ &\quad+P(T_z<\infty\mid T_z>T_y, T_y<\infty, X_0=x)P(T_z>T_y\mid T_y<\infty, X_0=x)\bigg] \\ &\quad\times P(T_y<\infty\mid X_0=x) \\ &=\bigg[P(T_z\leq T_y\mid T_y<\infty, X_0=x) \\ &\quad+P(T_z<\infty\mid T_z>T_y, T_y<\infty, X_0=x)P(T_z>T_y\mid T_y<\infty, X_0=x)\bigg] \\ &\quad\times P(T_y<\infty\mid X_0=x) \\ &\geq P(T_z<\infty\mid T_z>T_y,T_y<\infty, X_0=x)P(T_y<\infty\mid X_0=x) \\ &=P(T_z<\infty\mid X_0=y)P(T_y<\infty\mid X_0=x) \end{align*} $

The second line comes after omitting some positive number from the first line. The third line expands the first factor from the second line. The fourth line uses $P(T_z<\infty\mid T_z\leq T_y, T_y<\infty, X_0=x)=1$. The fifth line uses the inequality $a+b(1-a)\geq b$ when $a,b\in[0,1]$. The last line is the strong Markov property.