0
$\begingroup$

What are the possible measures of uncertainty for a discrete variable X=(x1, x2, ... xn), where values are defined by the alphabet - xi ∈ A, given probabilities p(xi) = P(X = xi) change over time?

E.g. consider time interval T∈( t0, tn) where t0 < t1 <... < tn
1) t = t0 < t1: P = { p(x0), p(x1), ..., p(x2) }
2) t = t1 < t2: P = { p(x0)', p(x1)', ..., p(x2)' }
3) ...

Where p(xi) ≠ p(xi)' and Δt = ti - ti-1 → 0

  • 0
    @Didier I think this answers my question, could you please post it as an answer so I can mark it?2011-07-20

1 Answers 1

1

Recall that a random variable X is not its image set A but a (measurable) function from a probability space (Ω,F,P) to a (measurable) set A. In your context, A={x1, x2, ... xn} and the distribution of the random variable X is characterized by a collection of nonnegative real numbers p(x1), p(x2), ..., and p(xn) summing to 1.

As @Chris said, your first task is to define a measure of uncertainty, suitable in your context. In other words, for every distribution p one must define a number U(p).

Two options are to use the variance

U(p)=M2(p)-M(p)2 with M2(p)=∑i xi2 p(xi) and M(p)=∑i xi p(xi),

or the entropy

U(p)=−∑i p(xi) log p(xi).

Both quantities are nonnegative, and zero only in the degenerate case when p(xi)=1 for a given i and p(xj)=0 for every other j.

Once the functional U is defined, if the distribution p changes with time t, one might consider the integral

0T U(p(t)) dt

of the functional U applied to p(t) on the time interval of interest.