I am trying to understand entropy. From what I know we can get the entropy of a variable lets say X.
What i dont understand is how to calculate the entropy of a matrix say m*n. I thought if columns are the attributes and rows are object we can sum the entropy of individual columns to get the final entropy(provided attributes are independent). I have couple of question
- IS my understanding right in case of independent attributes?
- What if the attributes are dependent? what happens to entropy? Is there where conditional entropy comes in?
Thanks