2
$\begingroup$

Given two densities $f$ and $g$ the squared Hellinger distance between them is defined as follows

$$ S(g,f)=H^2(g,f)=\frac{1}{2}\int_{\mathbb{R}}\left(\sqrt{g(y)}-\sqrt{f(y)}\right)^2 \mbox{d}y $$

I build a set

$$ {\cal{F}}=\{g:S(g,f)\leq \epsilon\} $$

with respect to the squared Hellinger distance and check if it is compact.

Question: Is this set ${\cal{F}}$ compact?

What I know:

$1$- ${\cal{F}}$ should be a convex set because the function that creates this set, $\left(\sqrt{g(y)}-\sqrt{f(y)}\right)^2$, is convex.

$2$- Hellinger distance satisfies the first three axioms of a metric http://en.wikipedia.org/wiki/Metric_%28mathematics%29 but not the triangle inequality (although I dont have any counter example, I read it in a paper which states it shortly in pharenthesis)

$3$- If ${\cal{F}}$ was a metric, I could say that it was compact because all continuous real functions of this set has a maxima.

Thanks in advance for reading this post.

  • 0
    nobody has any idea?2012-12-05
  • 0
    And what steps would you use to show it is NOT compact? Perhaps find an infinite sequence with no convergent subsequence...2012-12-06
  • 0
    @GEdgar I only know the definition of compactness with which I can not go one step further since it seems so abstract: each open cover of this set should have a finite subcover. So if there is one open cover which doesnt have a finite subcover it should mean that this set is noncompact but how to construct such open cover?2012-12-06
  • 0
    It seems to me that if all you know of compactness is the open-cover definition, then it is too soon for you to prove/disprove compactness in cases like this. You will need more study of general topology first. Probably some readers will provide proofs or disproofs as answers here, but (because of your background) you may not understand them.2012-12-06
  • 0
    @GEdgar I am aware of this. I will start learning them but first I must finish my own work. There is a paper and all the setup is the same except that the distance measure is Kullback-Leibler distance. Then the author says that the set is compact +..., therefore minimax rule can be used. He doesnt provide any details.2012-12-06
  • 0
    2- Even if the squared Hellinger distance is not a metric, its square-root $H(f,g) = \sqrt{H^2(f,g)}$ (the Hellinger distance) is.2012-12-06

2 Answers 2

1

Consider densities $f_n$ defined as follows. $f_n(x)=1$ for $0\le x \le 1/10$ and $n < x < n+1/10$, and $f_n(x)=0$ elsewhere. Then $H^2(f_n,f_m)$ is $1/10$ or some such thing, the same for all pairs $n \ne m$. So these $f_n$ all lie in set $\cal F$ for appropriate $\epsilon$. But no subsequence converges. So the set $\mathcal F$ is not compact.

1

The set $\mathcal{F}$ is not a compact set in the topology induced by this metric. To make a counterexample, take $f(x) = 1$ for $0 < x < 1$ and $0$ elsewhere and take $$ g_n(t) = (\delta + \epsilon \sin (2 \pi n t))^2 $$ for $n = 1, 2, \dots$, where $\delta = \sqrt{1 - \epsilon^2/2}$, where $\epsilon$ is sufficiently small. These are all probability densities. Then $H(g_n,f) < \epsilon$ while $H(g_n, g_m) = \epsilon/\sqrt{2}$ for all $n, m$. Hence the $g_n$ do not contain a convergent subsequence.

This can easily be modified to work also for densities that are everywhere positive on $\mathbb{R}$.

  • 0
    Thank you very much for the counterexample. As long as I understood, $H(g_n,g_m)=\epsilon/\sqrt{2}$ indicates that $g_n$ do not contain convergent subsequence? how do we understand this?2012-12-06
  • 0
    If there were a subsequence $g_k$ that converges to $g^\ast$, then for all $n$ $\lim_{k \to \infty} H(g_k,g_n) = H(g^\ast,g_n) = \epsilon/\sqrt{2}$. Therefore also $\lim_{n \to \infty} H(g^\ast,g_n) = H(g^\ast,g^\ast) = \epsilon/\sqrt{2}$. I guess that's impossible.2012-12-07
  • 0
    just one point: for all $n,m$? because $H(g_n,g_n)=0$.2012-12-07