In class I've recently learned how to construct from a discrete time Markov chain (DTMC) a continuous time Markov chain (CTMC) with the same transition probabilities by introducing exponential waiting times. As an exercise, I'm now trying to "reverse" this construction, i.e. constructing a DTMC from a given CTMC with the same invariant distribution.
Setting:
- an irreducible CTMC $X = (X_t: t\geq0)$ on a countable state space $S$ with invariant distribution $\pi$,
- a Poisson process $(N_t: t\geq0)$ with intensity $\lambda$ and inter arrival times $T_1, T_2, \dots$ that is independent of $X$.
Aim: Construct a DTMC with invariant distribution $\pi$.
Okay, my idea so far goes like this: The transition mechanism of $X$ depends on a continuous time parameter. To obtain a discrete chain $(Y_n)_{\in\mathbb N_0}$, we need to choose certain discrete points in time from which we extract the transition probabilities. Since we are given a Poisson process, it seems like a good idea to use the interarrival times $T_1, T_2, \dots$ which would yield the definition $$Y_n := X_{T_n}.$$ First of all: If $\left(P(t):t\geq0\right)$ is the semigroup of $X$, the transition matrix of $Y$ is $P(n)=P(T_n)$. So $Y$ is inhomogeneous, right?
Of course I should check whether $Y$ is actually Markov. Here I'm having some difficulty with the specifics. What I need to show is that $$\mathrm P(X_{T_{n+1}}=j \lvert X_{T_{n}}=j, X_{T_{n-1}}=i_{n-1}, \dots, X_{T_{0}}=i_0) = \mathrm P(X_{T_{n+1}}=j \lvert X_{T_{n}}=j)$$ This seems intuitively obvious since $X$ is Markov, the $T_k$ are memoryless and $X$ is independent of $(N_t)$. But I have a hard time coming up with a formal proof that really uses all of these properties. My first try was to apply the definition of conditional probability and use the law of total probability, but it didn't really lead anywhere.
Another thing that bothers me is where exactly irreducibility plays in.
I'd appreciate any help in figuring this out.