2
$\begingroup$

I am confused about the difference which the use of ";" and "," causes in the following expressions for defining mutual information between X and Y

$ I(X ; Y) $ and $ I(X , Y) $

  • 0
    There is no difference. The mutual information is defined between two random variables.2012-11-12

2 Answers 2

2

You often have $I(X_1,X_2,\ldots,X_n ; Y_1,Y_2, \ldots, Y_m)$. This is the mutual information between the vectors $(X_1,\ldots,X_n)$ and $(Y_1,\ldots,Y_m)$. The comma is used for vectors, the semicolon for separating the things between which the mutual information you're looking at.

This notation isn't confusing, since $I(X;Y;Z)$ doesn't define well (This is a standard homework problem from Cover & Thomas's Elements of Information Theory).

1

I believe it is because $I(X;Y) = I(Y;X)$. Mutual information does not measure a directional information flow, but how much $X$ can tell you about $Y$, and vice versa.

In addition, sometimes it is necessary to represent MI between $(X,Y)$ and $Z$, in which case $I(X,Y;Z)$ would be a sufficient notation.

  • 0
    For the last example, it would be good to mention that $I(X,Y;Z) \neq I(X;Y,Z)$, so writing e.g. $I(X,Y,Z)$ would not be well-defined.2014-02-22