3
$\begingroup$

I came across this question about a classic homework problem:

Let $(X_n)$ be i.i.d., positive random variables. Compute $$ E\left[\frac{\sum_{i=1}^k X_i}{\sum_{j=1}^n X_j}\right]$$ for $k \le n.$

In the question, the asker links to a previous answer to the same question and explains that their professor thought it was wrong, or inadequate in some way. I'm wondering why.

The answer, paraphrased, is

By linearity and symmetry, $$ E\left[\frac{\sum_{i=1}^k X_i}{\sum_{j=1}^n X_j}\right] = \sum_{i=1}^kE\left[\frac{X_i}{\sum_{j=1}^n X_j}\right] = kE\left[\frac{X_1}{\sum_{j=1}^n X_j}\right].$$ If $k=n,$ the answer is obviously $1,$ so we must have $$E\left[\frac{X_1}{\sum_{j=1}^n X_j}\right] = \frac{1}{n}.$$ Thus the answer is $\frac{k}{n}.$

They said their professor said this was not really a 'solution' and that they instead needed to condition on the denominator. Particularly, to define $M = \sum_{j=1}^nX_j$ and consider $$ E\left( \frac{\sum_{i=1}^k X_i}{M}\mid M = m\right)$$ where m is a positive integer. Then using the law of total probability or iterated expectation (as you can see in the two answers), the proof goes through much as before, using symmetry and linearity.

Nevermind the fact that there's no need for $M$ to be an integer. Let's take it for granted and assume that the $X_i$ are integer-valued RVs. (In fact the law of total probability answer only needs this assumption for convenience and the law of iterated expectations answer doesn't need it at all.)

What I don't get is how this conditioning improves the correctness of the solution at all. Can anyone think of a good reason?

Perhaps the professor forgot how linearity/numerators work and thought it was illegal to apply linearity until we'd pulled the denominator out of the conditional expectation? Or maybe there's some subtlety about the use of symmetry or linearity that I'm missing?

  • 1
    I think you're correct. The point is that the joint distribution of $(X_1,...,X_n)$ is invariant under permutations so $E[X_i / M]$ should be the same for all $i$. So $n$ times that is $1$ and thus it is $1/n$. Maybe someone can enlighten us if there is any fallacy. – 2017-02-09
  • 0
    I suppose a counter-example is when $X_i=0$ for all $i$. Or, more generally, when $X_i$ has a positive mass at 0. It might be better to assume these are all positive random variables, but, it is an interesting problem and clever proof. It looks like "i.i.d." can be weakened to "exchangeable." – 2017-02-09
  • 0
    @Michael ahh good call. I meant positive, not non-negative. The subtlety is in the assumption of existence of the expectation. And I agree that you probably only need exchangeability. It's an old problem I've seen done a few times. – 2017-02-09
  • 0
    @Michael Come to think of it, I'm not sure what suffices for the existence of that expectation. Probably just non-negative with no mass at zero like you say, since if they're all positive all RVs we're looking at the expectation of are always less than one. If you allow positive and negative you have problems. Pretty sure $Z_1/(Z_1 + Z_2)$ isn't integrable for, say, normals. – 2017-02-09

2 Answers 2

1

Exactly as you say:

$$\begin{align}\mathsf E\left(\frac{\sum_{i=1}^k X_i}{\sum_{j=1}^n X_j}\right) & = \sum_{i=1}^k\mathsf E\left(\frac{X_1}{\sum_{j=1}^n X_j }\right) \\[1ex] & = k~\mathsf E\left(\frac{X_1}{\sum_{j=1}^n X_j }\right)\end{align}$$

Your professor would want you to add:

$$\begin{align}&=k~\mathsf E\left(\mathsf E\left(\frac{X_1}{\sum_{j=1}^n X_j }~\middle\vert~ {\sum_{j=1}^n X_j }\right)\right)\\[1ex] & = k~\mathsf E\left(\frac{\mathsf E(X_1\mid \sum_{j=1}^n X_j)}{\sum_{j=1}^n X_j}\right)\end{align}$$

So that you argue that since obviously $\sum_{i=1}^{n}\mathsf E(X_i\mid \sum_{j=1}^n X_j)=\sum_{j=1}^n X_j$ , then by symmetry: $\mathsf E(X_1\mid \sum_{j=1}^n X_j)=\tfrac 1n\sum_{j=1}^n X_j$ and hence

$$\begin{align} &=k~\mathsf E\left(\frac{\sum_{j=1}^n X_j}{n~\sum_{j=1}^n X_j}\right) \\[1ex] &=\frac kn \\ \blacksquare & \end{align}$$

What these steps add to the proof is to make it much more obvious that symmetry argument can be applied.

Well, you can observe that $\mathsf E\left(\frac{\sum_{i=1}^n X_i}{\sum_{j=1}^n X_j }\right)=1$ so by symmetry it should be true that $\mathsf E\left(\frac{X_i}{\sum_{j=1}^n X_j }\right)=\frac 1n$ for all $X_i$.   It just leaves a little worm of doubt, at least in your professor's eyes, that you are justified to use symmetry at this point.   (Though the extra steps do make it clear that you are.)

  • 0
    To clarify, not my professor. The person who asked the linked question's professor. I guess obviousness is in the eye of the beholder, but I still don't see how it makes it better. What's wrong with the argument that $E(X_1/\sum X_i) =\frac{1}{n}$ since the answer must be $1$ when $k=n$? – 2017-02-09
  • 0
    Ok, I see you addressed my last question in your edit. – 2017-02-09
0

The critical point in your "proof" is that you need to show that the family of random variables $(Y_i)_{1\leq i\leq n}$ with $Y_i:=\frac{X_i}{\sum_{j=1}^{n}X_j}$ is identically distributed. In general, the fact that $(X_i)_{1\leq i\leq 2}$ are i.i.d. is not enough to say that $\frac{X_1}{Z}$ and $\frac{X_2}{Z}$ are identically distributed for a non-trivial random variable $Z$.

  • 0
    I think it is if $Z$ is symmetric in $X_1$ and $X_2,$ although you're right that one might want to justify this to be fully rigorous. However, how would conditioning improve this problem? By the same token, $E(X_1|Z)$ and $E(X_2|Z)$ are not necessarily identical just cause $X_1$ and $X_2$ are. The same justification would need to be made. – 2017-02-09
  • 0
    @spaceisdarkgreen: yeah exactly, that is why conditioning isn't needed. To elaborate on my above comment, you can define $f(x_1,...,x_n) = x_1 / \sum x_i$ and using the fact that product measures (of the same measure) are invariant under permutation, it follows that $f(X_1,...,X_n)$ has the same distribution as $f(X_{\sigma(1)},...,X_{\sigma(n)})$, for any permutation $\sigma$. This is a formal justification of the "symmetry" you are referring to, and it eliminates the need for conditioning. – 2017-02-09
  • 0
    @max Your point does however make it clear that the solution wouldn't go through if the assumption $k\le n$ were dropped, and how that assumption was used wasn't exactly transparent. (The result is false without it.) – 2017-02-09