I'm currently trying to evaluate an integral with respect to an MCMC kernel, and I wanted someone else to read what I `proved' to see if it's really correct. It just doesn't feel right.
Suppose I have an MCMC Kernel $K(a'|a)$ with stationary distribution $p(a)$. I want to take the expectation of a function $f(a)$ given a point $a'$ through the kernel like so,
$$ \int_{a} f(a) K(a|a') $$
I use detailed balance,
$$ = \int_{a} f(a) \frac{K(a'|a) p(a)}{p(a')} $$ $$ = \mathbb{E}[f(a)K(a'|a)] \frac{1}{p(a')} $$
Independence (at least I think they should be independent?) $$ = \mathbb{E}[f(a)] \mathbb{E}[K(a'|a)] \frac{1}{p(a')} $$
And detailed balance once again
$$ = \mathbb{E}[f(a)] p(a') \frac{1}{p(a')} $$
Now this just seems incorrect. If I interpret this correctly, it's telling me that if I take a large number of samples from $K(a|a')$ from any starting point $a'$ and integrate a function wrt to it, I might as well have had the original distribution. I must have done something wrong.