0
$\begingroup$

Consider a model with two binary stochastic nodes $x_0,x_1$ with probability of being on of $P(x_0), P(x_1)$ respectively.

When $x_i$ is in its on state it sends probability $u_0$ to some other variable, let's call it $y$.

We define the value of $y$ as being the product of all incoming probabilities, and if there are none then the value of $y$ is 1.0 (the null product).

In sum:

$y = \prod_{i:x_i = on} u_i \tag{1}$

A single stochastic sample of the model is given by:

$y{_k} = u_0^{x_0}u_1^{x_1} \tag{2}$

Where the $x_i$ are binary samples from the $P(x_i)$, and thus have values 0 or 1 for the off and on states respectively.

An approximation for the expected value of y can be obtained by averaging over a sufficiently large number of samples:

$\mathbb{E}[y] \approx \frac{1}{n} \sum_k y_k \tag{3}$

What is the exact expected value of y?

In this simple model we can obtain an exact value by enumerating over all possible combinations of the $x_i$, as follows:

$\mathbb{E}[y] = u_0 [P(x_0) - P(x_0)P(x_1)] + \\ u_1 [P(x_1) - P(x_0)P(x_1)] + \\ u_0 u_1 P(x_0)P(x_1) + \\ 1-P(x_0)P(x_1) \tag{4}$

However, the number of combinations to enumerate rapidly becomes impractical with increasing numbers of $x_i$. Are there efficient ways of finding $\mathbb{E}[y]$ or a good approximation?

  • 1
    Assuming I've understood correctly, if $y=\prod u_k^{x_k}$, it's just a product of $k$ independent random variables. As such, wouldn't we just have $\mathbb{E}[y]=\prod \mathbb{E}[u_k^{x_k}]$? – 2017-01-07
  • 0
    I considered that but it gives a different value compared to my numerical estimations, which is why I posted this Q. That equation works in sampling mode when the sampled $x_i$ are either 0 or 1, but it does not given the correct expected value when raising to the power of the probabilities directly. See my answer below for what I think is the correct approach. – 2017-01-07

1 Answers 1

0

While experimenting with specific sets of inputs for $P(x_0), P(x_1), u_0, u_1$ it seemed like $E[y]$ is simply the product of the expected values for the two partial products; and in fact this is the case.

I'll define the partial product from $x_i$ as $\hat x_i$, and its expected value is given by:

$\mathbb{E}[\hat x_i] = P(x_i) u_i + [1-P(x_i)]$

That is, it has value $u_i$ when $x_i$ is on ($x_i=1$), and value 1 when it is off ($x_i=0$).

Therefore the product of the two partial products $\hat x_0, \hat x_1$ is:

$ \prod_i P(x_i) u_i + [1-P(x_i)] $

Which for two $x_i$ expands out to be equivalent to equation (4) in the question text.

I.e. directly enumerating all of the combinations is not necessary as the required partial factors for each of the combinations are all represented by simple multiplication across all of the expected partial products.

  • 1
    Right: $\mathbb{E}(u_k^{x_k})=1-P(x_k)+P(x_k)u_k$, which matches up with my comment - this makes sense. – 2017-01-07