7
$\begingroup$

I am reading a book and having trouble following something. The problem is to try to maximize the differential entropy $-\int_{0}^{\infty}p(r)\log p(r)$ under the constraints that $\int_{0}^{\infty}p(r)=1$ and $\int_{0}^{\infty}rp(r)=\mu_{r}$. The book then just straightaway states that the result is an exponential distribution. I tried to prove this to myself but got stuck, it has been a while since I did any math so I am pretty sure I am missing something very basic.

Here's how far I have gotten. I first wrote the lagrangian

$L=-\int_{0}^{\infty}p(r)\log p(r)+\lambda_{1}\left(\int_{0}^{\infty}p(r)-1\right)+\lambda_{2}\left(\int_{0}^{\infty}rp(r)-\mu_{r}\right)\ \ \ \ \ \ \ \ (1)$

Now here is where I started getting a bit lost, I wasn't sure how to differentiate under the integral so I assumed that the integral was actually a sum over $p(r_{i})$ and differentiated $L$ w.r.t to $p(r_{i})$. Is this correct? What is the proper and general way to solve such differentials?

Moving along, after differentiation (assuming it was a sum) I got the following

$\frac{\partial L}{\partial p}=-\log p(r)-1+\lambda_{1}+\lambda_{2}r=0$

$\implies p(r)=e^{\lambda_{2}r+\lambda_{1}-1}\ \ \ \ \ \ \ \ (2)$

$\frac{\partial L}{\partial\lambda_{1}}=\int_{0}^{\infty}p(r)-1=0$

$\implies\int_{0}^{\infty}e^{\lambda_{2}r+\lambda_{1}-1}dr=1\ \ \ \ \ \ \ \ (3)$

$\frac{\partial L}{\partial\lambda_{2}}=\int_{0}^{\infty}rp(r)-1=0$

$\implies\int_{0}^{\infty}re^{\lambda_{2}r+\lambda_{1}-1}dr=1\ \ \ \ \ \ \ \ (4)$

But now as per my knowledge, the definite integrals in (3) and (4) should equal infinity, but that doesn't help me and so I am stuck? I am sure I am making some very basic conceptual mistake. Can someone please help me with this solution? Thanks.

  • 1
    definite integrals (3) and (4) are not necessarily equal to infinity. Consider $\lambda_2 = -1$ and $\lambda_1 = 1$.2011-05-06

2 Answers 2

3

$\int_0^\infty \mathrm e^{-a x} \mathrm d x = \frac{1}{a}$ for $a>0$.

$\int_0^\infty x \mathrm e^{-a x} \mathrm d x$ may be evaluated by integration by parts.

  • 0
    Thanks, I had forgotten my integrals. Just another silly doubt, seeing that I have $e^{\lambda_1 x}$ and not $e^{-\lambda_1 x}$ can I say let $\lambda_1=-c$ and then solve $\int_0^\infty \mathrm e^{-c x}$ and then later substitute back $\lambda_1$? Is there another better method?2011-05-06
2

Your question about the infinite integrals has already been answered (note that you can make your life easier by absorbing the $1$ into $\lambda_1$), so I'll just answer your question about how to systematically optimize such an integral. The general theory for this sort of thing is the calculus of variations, the main result of which is the Euler-Lagrange equation. However, since your integral depends only on $p$ and not on the derivative of $p$, the Euler-Lagrange equation just reduces to $\frac{\partial \mathcal L}{\partial p}=0$ (where $\mathcal L$ is the Lagrangian density, i.e. the integrand), so your calculation is correct.

  • 0
    Thanks, I did not know about calculus of variations.2011-05-06