Intuition behind Transition Kernels (without use of Markov Chains)

Question

I am studying Probability and a measure theory as a part of my masters course. I am very new to measure theory. I do understand the basic concepts but the one topic I am really struggling with Transition Kernels.

I am using the book "Probability and Stochastics" by Erhan Cinlar for reference.

I do not quite understand the idea behind Transition Kernels. In all the research I did on my own led me to the topic of Markov Chains and as of now I have very little knowledge of Markov Chains. Is there an easier way to define them? The below definition given on the book, is a little abstruse and so I would like to gain some intuition about it.

Given measure spaces (E,E) and (F,F) we define measure kernel K:E×F→[0,∞] such that:

i) x → K(x,B) is E measurable for every set B in F

ii) B → K(x,B) is a measure on (F,F) for every x in E

Any help is much appreciated!

Thanks in advance.

Answer 1

I would recommend first studying basic discrete time Markov chains without measure theory, as in my comment above.

---

To give a small amount of help in your current predicament: Basic discrete time Markov chains $\{M(t)\}_{t=0}^{\infty}$ over a finite or countably infinite state space $S$ are defined by a transition probability matrix $P=(P_{ij})$ such that $$P_{ij}=P[M(t+1)=j|M(t)=i] \quad \mbox{for all $i,j\in S$} $$

The crazy kernel stuff generalizes $P$ to uncountably infinite state spaces $S$. We have $$K(x,B)=P[M(t+1)\in B|M(t)=x] $$ for all $x\in S$ and all (measurable) subsets $B\subseteq S$. So $K$ defines a probability measure over the next-state, given the current state.

An example might be a state space $S$ given by the unit interval $[0,1]$. Given $M(t)=x$, define the next-state by $M(t+1)= (x + U_t) \mod 1$ (where the $\mod 1$ operation wraps the number back to the unit interval), where $\{U_t\}_{t=0}^{\infty}$ are i.i.d. random variables uniformly distributed over $[0,1/3]$. So, for example, given $x=1/3$, the kernel $K(1/3,\cdot)$ formally defines the next state according to a uniform distribution over $[1/3, 2/3]$. So: \begin{align} K(1/3, [2/3,1]) &= 0\\ K(1/3, [1/3, 1]) &= 1\\ K(1/3, [1/3, 1/2]) &=1/2 \end{align} and so on. Similarly, you can check that $K(5/6, [0,1/2]) = 1/2$.

---

Assuming the Markov states $\vec{M}(t)$ are vectors in $\mathbb{R}^n$, I personally would avoid the “kernel” stuff. I would just define the next-state transitions in terms of a conditional CDF function $$F(\vec{y}|\vec{x}) = P[\vec{M}(t+1) \leq \vec{y} | \vec{M}(t)=\vec{x}]$$ defined for all vectors $\vec{x}, \vec{y} \in \mathbb{R}^n$. And conditional PDFs can be defined as derivatives of the above (when differentiable). That is equivalent to specifying a kernel, yet it seems simpler. But that is just me.

Answer 2

I am also studying some basic Markov process on my own, and here is some of my basic understanding. The "naive" Markov process dealing with finite states can be well modeled by the transition matrices. But when one wants to generalize to infinite states and more general probabilisitic settings using measure theory, the kernel is what one needs. And instead of specifying the probability $\mathsf{Pr}(X_{n+1} = j| X_n = i)$ (like the element $P_{ij}$ in the transition matrix), here one specifies (informally) the possibility of being in some state among $J$, starting from the current state $i$: $\mathsf{Pr}(X_{n+1} \in J| X_n = i)$. And this leads to the definition of a kernel.

Let's consider a kernel $K : (E,\mathcal{E}) \to (E, \mathcal{E})$ (note that this kernel is on a single measurable space). Intuitively given $x\in E$ and $Y\in \mathcal{E}$, $K(x, Y)$ is the probability of arriving in some state in $Y$ from the current state $x$. Then the two conditions are natural. For example, to require that $\lambda Y. K(x, Y)$ is a probability measure since starting from $x$ is more or less the same as the requirement that in the transition matrix every row sum up to $1$. For some very simple examples one can refer to Markov kernel on IPFS.io.

By the way I am curious to see some simple examples where the source and target of the kernel are distinct. Anyone who has an example at hand or a reference, plz let me know!