Joint entropy of two random variables

Question

I flip a fair coin 4 times, and store the outcomes in a random variable X:

X = (X0, X1, X2, X3)

Then I have: Y = (X0, X1) and

X' = X3 * 8 + X2 * 4 + X1 * 2 + X0. Let Z = X' mod 6.

I have to compute the entropy H(Y,Z). I already computed the entropies H(X), H(Y) and H(Z) with the known entropy formula (the negative of the sum of the probability times the logarithm of the probability), but I have no idea how to compute a "joint entropy".

What is the way to go?

Answer 1

The idea is to combine all possible outcomes of Y (00, 01, 10, 11) and Z (0, 1, 2, 3, 4, 5) and put them in a table. Then, you compute all joint probabilities ( P((Y=00) AND (Z=0)), and so on. Then, you simply apply the formule of entropy.

The final is result is 3.5.