6
$\begingroup$

If a bunch of random variables $X_i$ are independently and identically distributed with an exponential distribution, their sum apparently follows a Gamma distribution.

But doesn't the central limit theorem imply that (for $X_i$ of any distribution with mean zero and variance $\sigma^2$), the sum $\sum_{i=1}^n X_i$ will become approximately normally distributed $~N(0,n\sigma^2)$ for large enough $n$ ?

Obviously I am missing something basic, but what's going on? How can the sum of i.i.d. exponential random variables have a Gamma distribution, but also be converging to normality?

2 Answers 2

7

There are several confusions here (I was also very confused when I started learning about that topic :-).)

  • Exponential random variables have a non zero mean (and are positive). The quantity you should be looking at, which asymptotically converges in distribution to a normal variable is $\sqrt{n} \left( \frac{\sum_{i = 1}^n X_i}{n} - \mu \right)$ The $\sqrt{n}$ was essential here, otherwise the distribution of the average will converge to a point mass at $\mu$. That quantity will converge to $N(0,\sigma^2)$. Both $\mu$ and $\sigma$ will be determined by the parameter of the exponential distribution.
  • The central limit theorem is asymptotic. The quantity $\sqrt{n} \left( \frac{\sum_{i = 1}^n X_i}{n} - \mu \right)$ will have a distribution. Let's call it $F_n$. (it is essential to remember that it depends on $n$). $F_n$ in general is not a normal distribution $N(0,\sigma^2)$. The central limit theorem tells us that that distribution gets in a certain sense closer and closer to $N(0,\sigma^2)$ as $n \to \infty$.
  • 0
    Ah, of course! That makes sense.2012-12-17
4

Good question! The Gamma distribution itself converges to a Normal distribution--exactly the Normal distribution that the Central Limit Theorem says the sum of the Exponential random variables converges to.

Of course, for any finite $n$ the distribution of the sum of Exponentials (the Gamma distribution) is not a normal distribution, since it's bounded below by zero, but as $n$ gets larger it gets closer and closer to a Normal distribution.

Here's a page that discusses the approximation, with some graphs that show how the Gamma distribution looks like a Normal when the first parameter (= the number of Exponential random variables you are summing up) gets large.

  • 1
    The shape of the Gamma will never be exactly a Normal distribution, even if you ignore the left-tail problem. But the CDF will get closer and closer to a Normal CDF.2012-12-17