2
$\begingroup$

Let's say I have some data, for example $d = 0.112$. And I have a known model $m$ which just produced uniformly distributed values over the interval $[0,.5]$. I am interested in computing the likelihood of my model given the data, in the Bayesian sense, i.e. $P(D=d | M = m)$.

What is this? is it $0$?

Given $m$ is a continuous distribution, I can't see how it could be anything other than likelihood 0? More generally, I can't see how the likelihood with any continuous model could be anything other than 0. I could imagine using the probability density function, but to be honest I am not entirely sure what this would mean, and it would give a likelihood of 2, which is greater than 1 and hence not a probability.

Thanks

  • 0
    Informally, we are in a "$0/0$" situation. More precisely, we look at intervals of small positive length, and then let that length approach $0$.2012-02-04

1 Answers 1

4

The simplest answer is that a likelihood function $\mathcal{L}$ is not a probability, which is why different names are used. In particular it does not need to sum or integrate to $1$.

For your uniform distribution, all of the values in $[0,0.5]$ are equally likely, so you want your calculation to produce the same likelihood for each of them. If you use the density function then you will get $2$ for all of them, but it would not matter if you came up with $10$ or any other constant positive number: it is the relative likelihood that matters.

This becomes even more obvious in Bayesian methods. If the prior distribution for the parameter $M$ is $\pi_0(m)$ and you use $\pi(m|D=d) = \frac{\pi_0(m) \mathcal{L}_{D=d}(M=m) }{ \int_n \pi_0(n) \mathcal{L}_{D=d}(M=n) \,dn}$ to calculate the posterior distribution for $M$ then it should be obvious that multiplying the likelihood function by a non-zero constant cancels out in the calculation of the posterior distribution.

  • 0
    @zenna: you haven't actually got a parameter of interest in your example. So let's suppose that your example has probability $m$ of being uniformly distributed on $[0,0.5]$ and $1-m$ of being normally distributed with mean $0.1$ and variance $0.2$; you then observe the value $0.3$. $\mathcal{L}(m)$, the likelihood of different values of $m$ given this data, is (proportional to) $2m+\dfrac{1-m}{\sqrt{0.4\, \pi }}\, e^{-0.1}$2012-02-06