I have a set of $M$ normally distributed random variables, $r_i$, each with an associated mean $u_i$, but the same variance $\sigma^2$. What is the variance of the average of these $M$ random variables: $\frac{\sum_{i=1}^{M} u_i}{M}$? How does the variance change as $M$ increases? What if the $M$ variables have a uniform, rather than a normal distribution, over some interval $[A, B]$?
The variance in the average of a set of normally distributed random variables
2
$\begingroup$
probability
random
normal-distribution
-
0And what did you try? – 2012-08-23
-
0What do you know about the sum of r.v. which are normally distributed? (I guess, $u_i$ are independent) – 2012-08-23
-
0@did Well, we know that the sum of randomly distributed normal variables is normally distributed, and that the mean and variance of this distribution is the sum of the means and variances of the random variables. Likewise, we know for uniformly distributed random variables, the mean of the sum is the sum of the means, and that the mean is going to be normally distributed by the central limit theorem. – 2012-08-23
-
0@Ilya The variables are independent, so the $u_i$ are also independent. We can also bound the $u_i$ over some interval. – 2012-08-23
-
0What does it mean for the $u_i$ to be _independent_? Are the $u_i$ themselves random variables, and thus the $M$ (unnamed) random variables really are _conditionally independent_ normal random variables with means $u_i$? – 2012-08-23
-
0@DilipSarwate: I am confused a bit whether $u_i$ are r.v. or means - from OP it is not clear at all. – 2012-08-23
-
0Sorry, the $u_i$ are the means of the random variables, which we can call $r_i$. – 2012-08-23
-
0If the means $u_i$ are _constants,_ then your statement that "the mean of the sum is the sum of the means, and that the mean is going to be normally distributed by the central limit theorem." makes no sense. The mean of the sum is a constant, and not a random variable. – 2012-08-23
1 Answers
2
Assuming the M variables are independent the average has a normal distrbution with mean equal to the average of the u$_i$s as you guessed and variance σ$^2$/M. The mean and the variance will be the same for a uniform but the average will have its distirbution on[A, B]. but if you define all the uniforms to be over the same interval [A, B] they will be IID and the distribution when the mean is appropriately normalized will converge to a normal by the central limit theorem.
-
1Yes but you have var([X1+X2+..+XM]/M)=[var(X1)+var(X2)+..+var(XM)]/M$^2$= M σ$^2$/M$^2$=σ$^2$/M. Note that var(cX)=c$^2$var(X). – 2012-08-23
-
0Why will the distribution of the average $\displaystyle \frac{1}{M}\sum_{i=1}^M X_i$ of $M$ random variables uniformly distributed on $[A,B]$ converge to a normal distribution? Doesn't the variance $\sigma^2/M = (B-A)^2/12M$ converge to $0$ as $M \to \infty$ and so the distribution of $\displaystyle \frac{1}{M}\sum_{i=1}^M X_i$ converges to a degenerate distribution at $(A+B)/2$? – 2012-08-23
-
0@DilipSarwate Of course the average converges a.s. to the true parameter value. I think you know that I meant to say "appropriately normalized". That should not have been omitted. Of course I am appealing to the CLT. – 2012-08-23
-
0The question is not whether I knew what you meant to say or whether knowledgeable readers of math.SE know that you were appealing to the CLT or not, but _whether the OP, who seems to be a beginning student of probability and statistics, has been given a precise answer. I think your "appropriately normalized" is still a waffling statement as far as the OP is concerned, though he has accepted your answer. – 2012-08-23
-
0@DilipSarwate The appropriate normalization involves multiplying by the square root of n. It is easy to find the statement of the central limit theorem in almost any book on statistics. I agreed that my statement needed qualification. My reference to you obviously knowing had to do with the way you made your comment and not to whether or not my statement would be clear to the general audience on this site. – 2012-08-23