1
$\begingroup$

Following is a part of an answer which was not resolved when I tried to answer a question in mathoverflow. I thought it would be nice to discuss that here.

Let $P$ and $Q$ be two distinct distributions on a finite set. For $0\le \lambda \le 1$, let $$L(\lambda)=D(P\|R_{\lambda})-D(Q\|R_{\lambda}),$$ where $$R_{\lambda}=\lambda P+(1-\lambda) Q.$$ I want to know the range of $\lambda$ for which $L(\lambda)\ge 0$.

I found that $\frac{d}{d\lambda}L(\lambda)=-\sum \frac{(P(a)-Q(a))^2}{R_{\lambda}(a)}\le 0$. So $L(\lambda)$ is decreasing. Also $L(0)=D(P\|Q)>0$ and $L(1)=-D(Q\|P)<0$. So at some $\lambda$, $L(\lambda)=0$.

The problem would be solved if we can find that $\lambda$. Is there a way to find it?

  • 0
    You could explain a reason why an expression for the solution $\lambda$ of $L(\lambda)=0$ should exist. I see none.2011-10-05
  • 0
    Isn't $L(\lambda)$ continuous in $\lambda$?2011-10-05
  • 0
    ?? And so what?2011-10-05
  • 0
    $L$ assumes positive value at $\lambda=0$ and negative at $\lambda=1$, so by intermediate value theorem, it must assume $0$ at some $\lambda$ in between, isn't it? Or am I lost completely? Sorry if I'm mistaken.2011-10-05
  • 0
    Sure, this argument is already in your post and is correct and shows that $L(\lambda)=0$ for some (unique) $\lambda$. But this is not your question, is it?2011-10-05
  • 0
    If I know that, say, it is $\lambda_1$ then for all $\lambda$ such that $0\le \lambda \le \lambda_1$, we have $L(\lambda)\ge 0$, isn't it?2011-10-05
  • 0
    Listen, the way this site works is that people are asking questions they want answers to, **in their posts**, not piece by piece in comments. Hence my advice would be to determine once and for all the exact question you want an answer to (something you seem not to have done) and to ask it in a post. At present there is no real question.2011-10-05
  • 0
    What does $\|$ mean?2011-10-05
  • 0
    For two probability distributions $P$ and $Q$ on a finite set $A$, $D(P\|Q)=\sum_{a\in A} P(a)\log \frac{P(a)}{Q(a)}$.2011-10-05
  • 0
    @ Didier Piau: Can you point what's not clear in the question? Aren't the things we have discussed in the comments very clear in question itself? I've gone through the question several times. I am really puzzled when you say there's no real question.2011-10-06
  • 0
    Apparently, you did not read my first comment. This comment does not ask why you think a root exists (the existence of a root is obvious) but why you think a simple expression for this root (presumably, as a function of the parameters $P(a)$ and $Q(a)$) should exist. I expressed my doubts that such an expression existed and since then, we have not gone further. // Unrelated: adding a space between the @ and the name of the person, like you did, ensures that the system DOES NOT signal your comment to the person (I read your comment by chance).2011-10-06
  • 0
    Sorry, as you said, I didn't understand your first comment carefully, so this confusion. Sorry.2011-10-07
  • 0
    Not using the @ thing altogether is another way to ensures the system does not signal your comment to the person it should be signaled to. My comments are signaled to you although I did not use any @ because you are the author of the question (this is the exception).2011-10-07

0 Answers 0