2
$\begingroup$

Here we have that $\pi$ is the invariant distribution of a Markov chain $K$. Let's say that $\mu=f\pi$, that is, $f$ is the density of $\mu$ and $\mu$ is a probability measure.

Why does $\mu K$ have density $K^* f$? I don't understand where the adjoint comes from because we're not dealing with any inner products here.

1 Answers 1

9

It seems part of the trouble here is caused by not defining properly the objects one is talking about, so let us try to be very careful about this.

Three kinds of objects are concerned: functions, measures and transition kernels. One can think about all of them as matrices of different sizes: for Markov chains defined on a finite state space of size $n$, functions have size $1\times n$, measures have size $n\times 1$ and transition kernels have size $n\times n$. Of course, for infinite state spaces, the dimension $n$ is infinite, nevertheless the interpretation as extended matrices is still useful.

For example, if $K$ is a kernel and $f$ is a function, $Kf$ has size $n\times n$ composed with $n\times 1$, this is size $n\times 1$ hence $Kf$ is a function, as it should be. Likewise, if $\mu$ is a measure, $\mu K$ has size $1\times n$ composed with $n\times n$, this is size $1\times n$ hence $\mu K$ is a measure, as it should be. And so on. Relevant formulas are $$ (Kf)(x)=\int K(x,\mathrm{d}y)f(y),\quad (\mu K)(\mathrm{d}x)=\int\mu(\mathrm{d}y)K(y,\mathrm{d}x). $$ Let us turn to your question. The setting is that $\pi$ is a stationary measure, that is, $\pi=\pi K$, or $$ \pi(\mathrm{d}x)=\int \pi(\mathrm{d}y)K(y,\mathrm{d}x), $$ and that $\mu$ has density $f$ with respect to $\pi$, that is, $$ \mu(\mathrm{d}x)=f(x)\pi(\mathrm{d}x). $$ We are concerned with the measure $\nu=\mu K$, that is, $$ \nu(\mathrm{d}x)=\int\mu(\mathrm{d}y)K(y,\mathrm{d}x)=\int f(y)\pi(\mathrm{d}y)K(y,\mathrm{d}x), $$ and we want to know the density $g$ of $\nu$ with respect to $\pi$, that is, we want to write $\nu$ as $$ \nu(\mathrm{d}x)=g(x)\pi(\mathrm{d}x). $$ The one and only possible function $g$ is $$ g(x)=\int f(y)\pi(\mathrm{d}y)\frac{K(y,\mathrm{d}x)}{\pi(\mathrm{d}x)}. $$ Hence $g=K^*f$ for a carefully chosen kernel $K^*$, namely $$ K^*(x,\mathrm{d}y)=\frac{K(y,\mathrm{d}x)}{\pi(\mathrm{d}x)}\pi(\mathrm{d}y). $$ One sees that $K^*$ is indeed the adjoint of $K$, as the notation suggests, in the following sense: for every functions $f_1$ and $f_2$ that are square integrable with respect to $\pi$, define their inner product in $L^2(\pi)$ by $$ \langle f_1,f_2\rangle_\pi=\int f_1(x)f_2(x)\pi(\mathrm{d}x). $$ Then, Fubini theorem yields $$ \langle f_1,K^*f_2\rangle_\pi=\int\int f_1(x)f_2(y)\pi(\mathrm{d}y)K(y,\mathrm{d}x)=\int Kf_1(y)f_2(y)\pi(\mathrm{d}y)=\langle Kf_1,f_2\rangle_\pi, $$ which is the definition of the adjoint of a linear operator.

  • 0
    THANK YOU! This helps so much---exactly what I needed.2011-03-20
  • 0
    You are welcome. Tried to help.2011-03-20
  • 0
    Really a very good answer2011-03-20
  • 0
    @TheBridge Thanks. Funny how everything becomes smooth and natural once one uses proper notations...2011-03-20
  • 0
    @Didier: Nice answer! I was referred here by your reply to my stationarity of Markov process question. I wonder about the way you construct the density g of ν with respect to π, by $g(x)=\int f(y)\pi(\mathrm{d}y)K(y,\mathrm{d}x) \frac1{\pi(\mathrm{d}x)}$. Is it part of Radon-Nykodym Theorem? Can I know a little more about this construction?2011-05-01
  • 0
    @Tim: Since $\pi=\pi K$, $K(y,\cdot)\ll\pi$ and one can read $K(y,\mathrm{d}x)/\pi(\mathrm{d}x)$ in the formula you refer to as the Radon-Nykodym density of $K(y,\cdot)$ with respect to $\pi$.2011-05-01
  • 0
    @Didier: Thanks! I was not asking how to go from that formula but how to get/construct it? Is it because $\nu(\mathrm{d}x)=g(x)\pi(\mathrm{d}x)$, and you moved $\pi(\mathrm{d}x)$ to the LHS just by division by it? Is $\frac1{\pi(\mathrm{d}x)}$ the ratio of 1 and $\pi(\mathrm{d}x)$ where $\mathrm{d}x$ is a measurable subset? Is this construction part of the proof of RN Theorem?2011-05-01
  • 0
    @Tim: $1/\pi(\mathrm{d}x)$ has no meaning but the ratio $K(y,\mathrm{d}x)/\pi(\mathrm{d}x)$ has as the value at $x$ of the Radon-Nykodym density of the measure $K(y,\cdot)$ with respect to $\pi$. I modified the typography of my post.2011-05-01
  • 0
    @Didier: Thanks! I think now I understand what you were saying. $K(y,\mathrm{d}x)/\pi(\mathrm{d}x)$ shouldn't be understood as an actual ratio/division, right?. It is just a notation for the density of $K$ wrt $\pi$2011-05-01