In my earlier question I asked about a technical aspect of solving a system of equations arising from looking for an entropy-maximizing distribution $p(x)$ continuous on $\mathbb{R}$ and constrained by KL-divergence with a zero-mean Gaussian distribution. That is, in addition to the usual probability density and variance constraints, I have the following constraint for $p(x)$:
$D(p_N(y)\|p(y))=\int_{-\infty}^{\infty}\frac{1}{\sqrt{2\pi}\sigma_N}e^{-y^2/2\sigma^2}\log\frac{\frac{1}{\sqrt{2\pi}\sigma}e^{-y^2/2\sigma^2}}{p(y)}dy<\epsilon$
Thanks to user anon, the form of the function $p(y)$ was found, but it is not a density function and now I am trying to interpret why is this the case.
First, here is the system of equations (copied from the earlier question) that I derived using Calculus of Variations (and help from Gallager's "Information Theory and Reliable Communication":
$\begin{align} 0&=\log(p(y))+1-\lambda-\gamma y^2-\eta \left(\frac{e^{-y^2/2}}{\sqrt{2\pi}}\right)\left(\frac{1}{p(y)}\right)\\ 0&=1-\int_{-\infty}^{\infty}p(y)dy\\ 0&=1-\int_{-\infty}^{\infty}y^2p(y)dy\\ 0&=c+\int_{-\infty}^{\infty}\frac{e^{-y^2/2}}{\sqrt{2\pi}}\log(p(y))dy \end{align} $
(for simplicity I set $\sigma=1$; $c=\epsilon+\frac{1}{2}\log(2\pi )$)
From anon's helpful comment, we can actually solve the first equation in terms of Lambert W function to obtain the following:
$p(x)=\frac{\eta e^{-y^2/2}}{\sqrt{2\pi}W(e^{-(1+2\gamma)y^2/2+(1-\lambda)})}$
When $|y|\rightarrow\infty$, $e^{-ay^2+b}\rightarrow 0$, and since $W(0)=0$, $p(y)\rightarrow\infty$. Thus, this is obviously not a pdf!
This is entirely due to the KL-divergence constraint (very similar situation arises when variance constraint is removed). How does one explain this? There are obviously probability distributions that meet the KL-divergence constraint (e.g. a Gaussian with appropriately picked variance). Does this mean that optimal distribution does not exist, and all distributions one can try would be sub-optimal? Is there a rigorous explanation for this?
Perhaps I did something wrong? Is there another method I should have employed?