1
$\begingroup$

After some calculations I ended up with (there each $V_k : \mathbb{R} \to \mathbb{R}$)

\sum_d \widetilde{P}_D(d) \left( \sum_{f' \in \mathcal{F}} P_{F|\;D}(f'|\;d) V_j(f',d) \right) \left( \sum_{f \in \mathcal{F}} P_{F|\;D}(f|\;d) V_i(f,d) \right)

Since

$ E_{P_{F|\;D}}[V_i|\;d] = \sum_{f \in \mathcal{F} } P_{F|\;D}(f|\;d) V_i(f,d) $

I get

$ \sum_d \widetilde{P}_D(d) E_{P_{F|\;D}}[V_j|\;d] E_{P_{F|\;D}}[V_i|\;d] $

Which I believe equals

$ E_{\widetilde{P}_d P_{F|\;D}}[V_i V_j] $

and just rewriting the first expression it is equal to:

\sum_d \widetilde{P}_D(d) \sum_{f \in \mathcal{F}} \sum_{f' \in \mathcal{F}} P_{F|\;D}(f|\;d) P_{F|\;D}(f'|\;d) V_j(f',d) V_i(f,d)

Which seems strange to me, that the expectation value of $V_i V_j$ would take into consideration the value when they are fed the same $d$ but different $f$.

I thought it would be written as

$ \sum_d \widetilde{P}_D(d) \sum_{f \in \mathcal{F}} P_{F|\;D}(f|\;d) V_j(f,d) V_i(f,d) $

Is it correct that the expression at the top equals $E_{\widetilde{P}_d P_{F|\;D}}[V_i V_j]$?

1 Answers 1

2

The trouble lies in the step where you write Which I believe equals, and comes from the fact that for some random variables $X$, $Y$ and $Z$, in general, $ \mathrm E(XY)\ne \mathrm E(\mathrm E(X\mid Z)\mathrm E(Y\mid Z)). $ For a counterexample, assume that $X$ and $Y$ are i.i.d. and square integrable random variables and that $Z=\frac12(X+Y)$.

The computations below hold in this general setting but one can assume for simplicity that the random variables $X$ and $Y$ are two independent centered $\pm1$ Bernoulli random variables or, equivalently, that $(X,Y)$ is uniform on the set $\{-1,1\}^2$. Then $\mathrm E(X)=\mathrm E(Y)=\mathrm E(Z)=0$ and $\mathrm E(Z^2)=\frac12$.

Then, on the one hand, $\mathrm E(X\mid Z)=\mathrm E(Y\mid Z)=\frac12\mathrm E(X+Y\mid Z)=Z$ by symmetry, hence $ \mathrm E(\mathrm E(X\mid Z)\mathrm E(Y\mid Z))=\mathrm E(Z^2). $ On the other hand, $\mathrm E(XY)=\mathrm E(X)\mathrm E(Y)$ by independence and $\mathrm E(X)=\mathrm E(Y)=\mathrm E(Z)$ by symmetry again, hence $\mathrm E(XY)=\mathrm E(Z)^2$. One sees that for every nondegenerate distribution of $X$ and $Y$, $ \mathrm E(\mathrm E(X\mid Z)\mathrm E(Y\mid Z))-\mathrm E(XY)=\text{Var}(Z)\ne0. $ Edit For a case where $\mathrm E(\mathrm E(X\mid Z)\mathrm E(Y\mid Z))-\mathrm E(XY)\lt0$, consider $Z=\frac12(X-Y)$.

  • 0
    Many thanks for this.2012-01-24