2
$\begingroup$

Given two densities $f$ and $g$ the squared Hellinger distance between them is defined as follows

$ S(g,f)=H^2(g,f)=\frac{1}{2}\int_{\mathbb{R}}\left(\sqrt{g(y)}-\sqrt{f(y)}\right)^2 \mbox{d}y $

I build a set

$ {\cal{F}}=\{g:S(g,f)\leq \epsilon\} $

with respect to the squared Hellinger distance and check if it is compact.

Question: Is this set ${\cal{F}}$ compact?

What I know:

$1$- ${\cal{F}}$ should be a convex set because the function that creates this set, $\left(\sqrt{g(y)}-\sqrt{f(y)}\right)^2$, is convex.

$2$- Hellinger distance satisfies the first three axioms of a metric http://en.wikipedia.org/wiki/Metric_%28mathematics%29 but not the triangle inequality (although I dont have any counter example, I read it in a paper which states it shortly in pharenthesis)

$3$- If ${\cal{F}}$ was a metric, I could say that it was compact because all continuous real functions of this set has a maxima.

Thanks in advance for reading this post.

  • 0
    2- Even if the squared Hellinger distance is not a metric, its square-root $H(f,g) = \sqrt{H^2(f,g)}$ (the Hellinger distance) is.2012-12-06

2 Answers 2

1

Consider densities $f_n$ defined as follows. $f_n(x)=1$ for $0\le x \le 1/10$ and $n < x < n+1/10$, and $f_n(x)=0$ elsewhere. Then $H^2(f_n,f_m)$ is $1/10$ or some such thing, the same for all pairs $n \ne m$. So these $f_n$ all lie in set $\cal F$ for appropriate $\epsilon$. But no subsequence converges. So the set $\mathcal F$ is not compact.

1

The set $\mathcal{F}$ is not a compact set in the topology induced by this metric. To make a counterexample, take $f(x) = 1$ for $0 < x < 1$ and $0$ elsewhere and take $ g_n(t) = (\delta + \epsilon \sin (2 \pi n t))^2 $ for $n = 1, 2, \dots$, where $\delta = \sqrt{1 - \epsilon^2/2}$, where $\epsilon$ is sufficiently small. These are all probability densities. Then $H(g_n,f) < \epsilon$ while $H(g_n, g_m) = \epsilon/\sqrt{2}$ for all $n, m$. Hence the $g_n$ do not contain a convergent subsequence.

This can easily be modified to work also for densities that are everywhere positive on $\mathbb{R}$.

  • 0
    just one point: for all $n,m$? because $H(g_n,g_n)=0$.2012-12-07