Confused on the general addition rule for variances.
Why is it that when two random variables, X and Y, have a perfect positive correlation (p=1) their standard deviations add. But when they are uncorrelated (p=0) their variances add?
Confused on the general addition rule for variances.
Why is it that when two random variables, X and Y, have a perfect positive correlation (p=1) their standard deviations add. But when they are uncorrelated (p=0) their variances add?
The equation $\text{Var}(X+Y) = \text{Var}(X) + \text{Var}(Y) + 2 \text{Cov}(X,Y)$ covers all cases. Note that $ \text{Cov}(X,Y) = \rho\; \sigma_X\; \sigma_Y$.
Where it comes from is the linearity of expected value:
$\eqalign{E[X+Y] &= E[X] + E[Y]\cr E[(X+Y)^2] &= E[X^2 + 2 X Y + Y^2] = E[X^2] + 2 E[XY] + E[Y^2]\cr}$
and the definitions of variance and covariance in terms of expected values.
Further illumination on Robert Israel's answer:
Starting with the formula $ \begin{align} \mathrm{Var}(X) &=\mathrm{E}\left((X-\mathrm{E}(X))^2\right)\\ &=\mathrm{E}(X^2)-\mathrm{E}(X)^2\\ \end{align} $ we get $ \begin{align} \mathrm{Var}(X+Y) &=\mathrm{E}\left(X^2+2XY+Y^2\right)-\left(\mathrm{E}(X)^2+2\mathrm{E}(X)\mathrm{E}(Y)+\mathrm{E}(y)^2\right)\\ &=\left(\mathrm{E}(X^2)-\mathrm{E}(X)^2\right)+\left(\mathrm{E}(Y^2)-\mathrm{E}(Y)^2\right)+2\Big(\mathrm{E}(XY)-\mathrm{E}(X)\mathrm{E}(Y)\Big)\\[5pt] &=\mathrm{Var}(X)+\mathrm{Var}(Y)+2\mathrm{Cov}(X,Y) \end{align} $ Furthermore, using Hölder, $ \begin{align} \left|\mathrm{Cov}(X,Y)\right|^2 &=\left|\mathrm{E}\Big((X-\mathrm{E}(X))(Y-\mathrm{E}(Y))\Big)\right|^2\\ &\le\mathrm{E}\left((X-\mathrm{E}(X))^2\right)\mathrm{E}\left((Y-\mathrm{E}(Y))^2\right)\\ &=\mathrm{Var}(X)\mathrm{Var}(Y) \end{align} $ For perfect positive correlation, we have $\mathrm{Cov}(X,Y)=\mathrm{Var}(X)^{1/2}\mathrm{Var}(Y)^{1/2}$; therefore, $ \begin{align} \mathrm{Var}(X+Y) &=\mathrm{Var}(X)+\mathrm{Var}(Y)+2\mathrm{Var}(X)^{1/2}\mathrm{Var}(Y)^{1/2}\\ \mathrm{Std}(X+Y)^2 &=\mathrm{Std}(X)^2+\mathrm{Std}(Y)^2+2\mathrm{Std}(X)\mathrm{Std}(Y)\\ \mathrm{Std}(X+Y) &=\mathrm{Std}(X)+\mathrm{Std}(Y) \end{align} $
For independence, we have $\mathrm{Cov}(X,Y)=0$; therefore, $ \mathrm{Var}(X+Y)=\mathrm{Var}(X)+\mathrm{Var}(Y) $