Conditional probability of an event

Question

Why is $P(\Theta \mid D,M) = P(D \mid \Theta,M)\frac{P(\Theta \mid M)}{P(D \mid M)}$?

It is obvious that this equation is true, but why? I can't find a mathematical proof.

Answer 1

Just use the definition of conditional probability.

$\begin{align}\mathsf P(\Theta\mid D,M) &= \dfrac{\mathsf P(\Theta, D, M)}{\mathsf P(D,M)} \\[1ex] &= \dfrac{\mathsf P(D\mid \Theta, M)~\mathsf P(\Theta, M)}{\mathsf P(D,M)} \\[1ex] &= \dfrac{\mathsf P(D\mid \Theta, M)~\mathsf P(\Theta\mid M)~\mathsf P(M)}{\mathsf P(D\mid M)~\mathsf P(M)} \\[1ex] &= \dfrac{\mathsf P(D\mid \Theta, M)~\mathsf P(\Theta\mid M)}{\mathsf P(D\mid M)} \end{align}$

Answer 2

$P(A \land (B \land C)) = P(A|B , C) * P(B \land C)$

Likewise:

$P(B \land (A \land C)) = P(B|A , C) * P(A \land C)$

But of course

$P(A \land (B \land C)) = P(B \land (A \land C))$

So:

$P(A|B , C) * P(B \land C) = P(B|A , C) * P(A \land C)$

But since

$P(B \land C) = P(B|C)*P(C)$ and $P(A \land C) = P(A|C)*P(C)$,

we get:

$P(A|B , C) * P(B|C)*P(C) = P(B|A , C) * P(A|C)*P(C)$

because $P(C) \not = 0$, we can divide both sides by $P(C)$:

$P(A|B , C) * P(B|C) = P(B|A , C) * P(A|C)$

and so:

$$P(A|B , C) = \frac{P(B|A , C) * P(A|C)}{P(B|C)}$$

Answer 3

This is basically Bayes' theorem, applied on the same probability space, but with a different probability measure - the conditional probability given $M$.

In other words, suppose we start with a probability space $(\Omega,\Sigma,P)$. Now, let $P_M$ be a different probability measure, defined by $P_M(A) = P(A|M)$. Then we get a new probability space $(\Omega,\Sigma,P_M)$. In this probability space, the Bayes' theorem holds, as with any other probability space.

Then, $P(\Theta \mid D,M) = P(D \mid \Theta,M)\frac{P(\Theta \mid M)}{P(D \mid M)}$ is equivalent to: $$P_M(\Theta \mid D) = \frac{P_M(D \mid \Theta)P_M(\Theta)}{P_M(D)},$$ which is the statement of the Bayes' theorem and is easily proven.

The result could of course be derived easily by definition, as others have done. Maybe this is a bit of an overkill, but I just wanted to give another way to look at the problem and to note that by taking conditional probability, you in a way restrict the probability space you are working with, and everything you know about probability applies to this space. Maybe this helps you gain some more intuition with working with conditional probability.