7
$\begingroup$

I am trying to understand entropy. From what I know we can get the entropy of a variable lets say X.

What i dont understand is how to calculate the entropy of a matrix say m*n. I thought if columns are the attributes and rows are object we can sum the entropy of individual columns to get the final entropy(provided attributes are independent). I have couple of question

  1. IS my understanding right in case of independent attributes?
  2. What if the attributes are dependent? what happens to entropy? Is there where conditional entropy comes in?

Thanks

2 Answers 2