I've always used Shannon's entropy for measuring uncertainty, but I wonder why to use a logarithmic approach. Why shouldn't uncertainty be linear?
For instance, consider the following pairs of distributions:
$ \left[0.9, 0.1\right], \left[0.99, 0.01\right] $
$ \left[0.4, 0.6\right], \left[0.49, 0.51\right] $
Then you have the following uncertainty measures: $ H([0.9, 0.1]) = 0.46899559358928122\\ H([0.99, 0.01]) = 0.080793135895911181\\ H([0.4,0.6]) = 0.97095059445466858\\ H([0.49, 0.51]) = 0.9997114417528099 $
With the following example I just want show that it doesn't satisfy linearity:
$ H([0.9, 0.1]) - H([0.99, 0.01]) \simeq 0.3882 $
$ H([0.49, 0.51]) - H([0.4, 0.6]) \simeq 0.02876 $
As we can see, distributions that are closer to $[1,0]$ or $[0,1]$ tend faster to zero.
May be this is more a philosophic question but I think that may be someone could give me alternative measures of uncertainty that may be linear or, at least, provide some explanation to the rationale of this approach.
Thanks!
EDIT I don't mean linearity in the whole space but in the intervals $[0,\frac{1}{2}]$ and $[\frac{1}{2}, 1]$. Since, as @r.e.s. comments, is a required property for such a measure that $f(0)