1
$\begingroup$

I have some basic questions on the practical meaning of Markov Chain Monte Carlo (MCMC) methods, as, for example, the Gibbs sampling

Suppose I have a random matrix $\underbrace{Y}_{n\times n}$ defined on the probability space $(\Omega, \mathcal{F}, \mathbb{P})$ with support $\{A_i\}_{i=1}^3$, $A_i\in \mathbb{R}^n$ for $i=1,2,3$.

Additionally, $\mathbb{P}(Y=A_i)=f(A_i,\theta)$ where $\theta\in \Theta\subseteq \mathbb{R}$ for $i=1,2,3$.

Assume $\theta$ is known.

Suppose that $f(A_i;\theta)$ is particularly hard to compute for $i=1,2,3$. One way to overcome this issue is by simulating it using a MCMC method:

1) I start with a certain realisation of $Y$

2) I update $Y$ using a certain transition probability (suppose I am able to compute this)

3) I repeat step 2) $T$ times with $T$ very large

The outcome of this algorithm is for example $$ A_1, A_2, A_1, A_3, A_1, A_1, A_1, A_2,..., A_3 $$

(*) The magic is that if I plot the probability mass function of $$ A_1, A_2, A_1, A_3, A_1, A_1, A_1, A_2,..., A_3 $$ I get something very close to $f(A_1;\theta)$, $f(A_2;\theta)$, $f(A_3;\theta)$. Moreover if I compute the average of $$ A_1, A_2, A_1, A_3, A_1, A_1, A_1, A_2,..., A_3 $$ I get something very close to $E(Y)$.

Questions: is my summary above correct?

I am confused on the words often used in other sources to state (*), such as "the distribution of $Y$ converges to $f$ for $T\rightarrow \infty$", or "the sampler converges to a stable (stationary) distribution", etc. Are these equivalent ways to state (*)?

What is meant by "stationary distribution"? Stationary reminds me something that does not change but I don't see the relation here.

  • 0
    The "magic" and the "certain transition probability" are intimately connected, you need to find a transition probability for your updates so that when you pick some subsequence $Y_{n_1},Y_{n_2},\ldots$ is equivalent to a sequence of random variables from the distribution $f$, once your chain has achieved a (effectively) stationary state then the distribution becomes invariant - which is where the idea of "does not change" occurs – 2017-02-22
  • 0
    Thanks. I understand that getting the transition probability is complicated, but here I am asking about the output of MCMC methods. Again, what does it mean that "your chain has achieved a (effectively) stationary state"? From how it is written it seems to indicate that after a certain number of iterations, the realisation of $Y$ that I get from the algorithm does not change anymore, which I think it is not the correct interpretation. – 2017-02-22
  • 0
    Getting the transition probability doesn't have to be that complicated, standard methods like Metropolis-Hastings or Gibbs sampling are often relatively easy to implement, you have some target distribution, $f$ say, in mind but for whatever reason you can't sample from it so instead you construct a different random process such that you can show the long run behaviour is that of the target distribution. – 2017-02-22
  • 0
    @Nadiels Thanks, but, again, this is not what I'm asking about, this question is not about transition probabilities. – 2017-02-22
  • 0
    Say for example that your density $f$ had some intractable normalising say $f(x) = Z h(x)$, now a Metropolis-Hastings approach to simulating this random variable would consist of a *proposal distribution* $p(x_{new} | x_{cur})$ and an acceptance ratio $A =\frac{f(x_{new})p(x_{cur} | x_{new}) }{f(x_{cur}) p(x_{new} | x_{cur})}$ so that difficult normalising constant disappears in the ratio - that's just one example where it works, maybe fill out your intended application a little more? – 2017-02-22
  • 0
    you say it isn't about the transition probabilities, but it is because you cannot ask about the stationary state and not be asking about the transition probabilities... someone else can have a better go at answering what you're asking I guess. – 2017-02-22

0 Answers 0