I've been implementing logarithmic number system and I came across these functions called Gaussian logarithms:
- $f(x) = \log(1 + e^x)$.
- $g(x) = \log(e^x - 1)$ for $x > 0$.
- $h(x) = \log(1 - e^x)$ for $x < 0$.
Since I am lazy, I assume $\log$ and $\exp$ aren't expensive to compute, but $\exp$ of a very big number causes a problem because I'm using double
. So, I found, with some guess work, nice approximations of the above functions:
- $\bar f(x) = x + e^{-x}$ for $x$ large.
- $\bar g(x) = x - e^{-x}$ for $x$ large.
- $\bar h(x) =$ ... Actually this one is not needed because $\exp(x)$ is small for negative $x$. (You see how lazy I am.)
It turns out that $\bar f$ and $\bar g$ are surprisingly good estimates of $f$ and $g$. I can justify why they are decent estimates as follows:
$f(x) = \log(e^x(1 + e^{-x})) \approx \log(e^x(e^{e^{-x}})) = x + e^{-x}$.
Similarly, $g(x) = \log(e^x(1 - e^{-x})) \approx \log(e^x(e^{-e^{-x}})) = x - e^{-x}$.
However, as I try to do a more careful analysis, I get stuck:
$f(x) = \log(e^x(1 + e^{-x})) = \log(e^x(e^{e^{-x}}) + e^xO(e^{-2x}))) = ???$
What is a proper way to do an analysis like this?
Here's my initial attempt: The derivative of $\log(z)$ is bounded by $1$ for $z \ge 1$, so
$f(x) = x + \log(e^{e^{-x}} + O(e^{-2x})) \approx x + e^{-x} + O(e^{-2x})$,
but I have a feeling that this method of analysis is not very good because ... it's so unfamiliar. It requires the knowledge that $e^{e^{-x}} \ge 1$ in order to get the bound, and the bound seems pretty loose in my opinion. Also, I don't see a way to improve the estimate by "adding more terms" like what I could do with Taylor series.
I'd really appreciate if someone can show me a more standard method of analyzing this situation. And it would be really nice if more accurate estimates or an arbitrary precision method can be derived.