14
$\begingroup$

I've seen the following formula being used in various places:

$P(a|b) = \sum_z P(a|z) \times P(z|b)$

So essentially they sum over all the variables z to get the original conditional probability. Is this correct, and does it make any strong assumptions?

I am confused by the following example:

$P(good\ weather | month) = \\ \sum_{z \in \{T,F\}} P(good\ weather| universe\ exists = z) * P(universe\ exists = z| month) = \\ P(good\ weather|universe\ exists = T) $

It starts off with the probability of having good weather, given the current month. We then introduce an additional variable (does the universe exist), and sum over all values (T,F). Let's say that the universe exists with probability 1.0, then:

$P(universe\ exists = T| month) = 1.0$

$P(universe\ exists = F| month) = 0.0$

Therefore, we end up with:

$P(good\ weather | month) = P(good\ weather | universe\ exists = T)$

Now, somewhere I have clearly lost some important information, because I'm showing that the conditional probability of good weather, depending on the month, is equal to the probability of good weather, given that the universe exists. What am I missing?

1 Answers 1

14

The equation does certainly not apply in general. What you can write however is \begin{equation} P(a|b) = \sum_z P(a,z|b), \end{equation} which is sometimes referred to as marginalization. Note that the above equation simply describes how to go from a joint probability mass function $P(x,y)$ to the probability mass function $P(x)$ (or $P(y)$), that is, by summing out the other variable. (Similar arguments apply to probability density functions by replacing the above summation with integration.)

Now, one can apply Bayes rule and further expand $P(a,z|b) = P(a|z,b) P(z|b)$ to get\begin{equation} P(a|b) = \sum_z P(a|z,b) P(z|b). \end{equation} The formula that you posted simply drops the conditioning on $b$ in the first factor (implying $P(a|z,b) = P(a|z)$), which cannot be done in general.

  • 0
    You are welcome. I edited the typo you mentioned.2012-11-14