Calculate covariance for discrete random variables

Question

This question might seem quite easy for many of you, however, I think I need a little help in the right direction.

I'm currently reading about probability theory and have come across covariance. I know the definition of covariance and I'm trying to solve some exercises.

For instance, I have been given a discrete random variable X with probability function px(x) = 1/2 if x = -1, 1/4 if x = 0, 1/4 if x = 1, 0 otherwise.

Additionally, I have been given a discrete random variable Y, which is independent of X, and has probability function py(y) = 3/4 if y = 0, 1/4 if y = 1, 0 otherwise.

I then have to calculate Cov(Y, 2Y - X). The answer is given and should be 3/8.

What confuses me a bit when I'm reading about covariance is that some of the formulas I have come across uses pairs consisting of X values and Y values, but for example in this exercise there are three X values and only two Y values.

Furthermore, when two discrete random variables X and Y are independent, which this exercise says (it says Y is independent of X), then Cov(X, Y) should be equal to 0. But when I use the rule E(X * Y) = E(X) * E(Y) for independent variables, I, however, end up with a formula indicating that the result should be 0 and not 3/8.

For example when I try to deduce from the definition Cov(X, Y) = E[(X - E[X])(Y - E[Y])] I get the following:

Cov(Y, 2Y-X) = E[(Y - E[Y])((2Y - X) - E[2Y - X])]

This can be rewritten using Cov(X, Y) = E[X * Y] - E[X] * E[Y] which I think is more manageable:

Cov(Y, 2Y-X) = E[Y * (2Y - X)] - E[Y] * E[2Y - X]

Since X and Y are independent, I should be able to use the rule E(X * Y) = E(X) * E(Y) so that I get:

Cov(Y, 2Y-X) = E[Y] * E[2Y - X] - E[Y] * E[2Y - X]

which indicate the result is 0, which is wrong according to the result given.

So if we say I'll just try to proceed with this one:

Cov(Y, 2Y-X) = E[Y * (2Y - X)] - E[Y] * E[2Y - X]

How do I continue from here? I know how to calculate the expected value of Y for example, but how do I calculate the expected value of 2Y - X? The formula I found for this suggested using pairs of X values and Y values, but I don't know how to do that when there aren't the same amount of X values and Y values.

Answer 1

Where you went wrong is in the step where yu say $X$ and $Y$ are independent (true) but use that as if you had said $2Y-X$ and $Y$ are independent (false). Surely if you specify the value of $Y$, that gives you some information about the value of $2Y-X$.

So you need to start with the equation you said you would like to proceed with: $$ \mbox{cov }(Y,2Y-X) = E[ Y(2Y-X)] -E[Y]E[2Y-X] \\ \mbox{cov }(Y,2Y-X) = 2E[ Y^2] - E[YX] - E[Y]E[X] $$ Now with your probability distribution as given: $$ E[Y] = \frac34(0) + \frac14(1) = \frac14\\ E(Y^2) = \frac34(0^2) + \frac14(1^2) = \frac14\\ E[X] = \frac12(-1)+\frac14(0)+\frac14(1) = -\frac14\\ E[YX]=\frac34\frac12(0\cdot (-1)) + \frac34\frac14(0\cdot (0)) + \frac34\frac14(0\cdot (1)) \\+ \frac14\frac12(1\cdot (-1)) + \frac14\frac14(1\cdot (0)) + \frac14\frac14(1\cdot (1)) = -\frac18+\frac1{16} = -\frac1{16} $$ And then $$ \mbox{cov }(Y,2Y-X) = 2E[ Y^2] - E[YX] - E[Y]E[X] \\= 2\cdot \frac14 - \left(-\frac1{16}\right)- \frac14 \cdot \frac14 = \frac12-2 \left(\frac1{16}\right) = \frac38 $$

Answer 2

You are correct that $$Cov(Y, 2Y-X) = E[Y * (2Y - X)] - E[Y] * E[2Y - X].\tag1 $$ However, it's not true that $Y$ and $2Y-X$ are independent, so you cannot say $$E[Y * (2Y - X)]=E(Y) E(2Y-X).$$ The correct way to proceed with (1) is to multiply out $Y(2Y-X)$ to obtain $2Y^2-XY$. The expectation of this is $2E(Y^2)-E(XY)$. Now you can apply independence of $X$ and $Y$ to write $E(XY)=E(X)E(Y)$. At this point (1) reduces to calculating quantities like $E(X)$, $E(Y)$ and $E(Y^2)$, which you have enough information to do.

Aside: If you want to calculate covariance using pairs of $X$ and $Y$ values, you can do that: The independence between $X$ and $Y$ specifies the probability of any pair of $X$ and $Y$ values, as follows: $$P(X=x,Y=y)=P(X=x)P(Y=y).$$