The intuition behind gamma distribution

Question

What is the intuition behind gamma distribution?

For instance, I understand how to "construct" Gaussian distribution. This is my intuition:

1. Bernoulli distribution - which is simple concept
2. A sequence of Bernoulli trials is a Binomial distribution. I understand how binomial coefficient is constructed
3. Using Stirling approximation we can deduce Gaussian distribution

Hence I understand that shape of a Gaussian distribution is determined by the binomial coefficients and so on.

How can Gamma distribution be derived step by step using relatively simple concepts?

Answer 1

The short answer is "prove the pdf is non-negative & integrates to 1", since that guarantees we're dealing with a probability distribution. But if you're looking for a motivation for why anything would be Gamma-distributed, here's the idea.

The $\alpha =1$ special case, with pdf $e^{-\beta x}$ for $x>0$, is the so-called exponential distribution, which concerns the lifetime of something that has the same risk of failure in a given time period regardless of its history (we call this memorylessness), e.g. a radioactive atom. (The mean lifetime is $\beta^{-1}$.) If $\alpha$ is a positive integer, you have the distribution obtained by summing $\alpha$ exponential iids. In general $\alpha$ can be any positive real, but that's just a continuous generalisation.

Answer 2

I understand the exponential distribution in a following way:

$$e^{-\lambda \cdot x} = \lim_{N \to \infty} (1 - \frac{\lambda}{N})^{N \cdot x}$$

Where:

$\lambda$ - number of events per one time unit (denoted by $1$)

$N$ - tends to infinity. It splits whole time unit into $N$ small intervals each of a length of $\frac{1}{N}$, such that only one event can occur within this small interval

$\frac{\lambda}{N}$ - is a probability of an event within one small time frame. Each time frame is a Bernoulli trial: event - success, no event - failure.

$(1 - \frac{\lambda}{N})$ - probability of "failure" - no event

$N \cdot x$ - is a number of consecutive "failures". Where $x$ is a part of 1 interval, let $N = 1000$ - some big number, then half of the interval ($x = 0.5$) would be $N \cdot x = 1000 * 0.5 = 500$ small time frames

$(1 - \frac{\lambda}{N})^{N \cdot x}$ - probability of $N \cdot x$ consecutive failures, or probability that the event will not occur $x$ amount of time