Suppose that $X_1, X_2,\cdots, X_n$ are iid random variables from $N(\theta,1)$, $\theta$ is rational. Then we know that $\bar X \sim N(\theta,1/n)$. It is said that $\bar X$ is almost surely irrational. I am wondering why is it irrational a.s.? How can I interpret it?
why is this $\bar X$ a.s. irrational?
1
$\begingroup$
statistics
probability-distributions
1 Answers
4
Let me denote $\bar X$ by $Z$. Since $Z$ is a random variable with continuous density, the probability of the event $E_q$ that $Z = q$ is $0$ for any fixed real $q$. Now, the event $Z \in \mathbb Q$ can be written as $\bigcup_{q \in \mathbb Q} E_q$; hence by countable additivity, this event also has a zero probability.
We effectively ignored a lot of the details given in the question (e.g., $Z$ is normally distributed, that $\theta$ is rational). The conclusion does not change as long as $Z$ has a continuous distribution.
-
0I see, thus if we restrict $\theta\in \mathbb{R}\backslash\mathbb{Q}$, we can have that the mle of $\theta$ is $\bar X$ a.s. Then I am wondering what if we restrict $\theta\in \mathbb{Q}$? Can we still find an mle for $\theta$? – 2011-11-12
-
0@Zoe Please wait for 1-2 days before accepting an answer. Accepting an answer too soon might discourage other users from posting their answers. (Regarding your comment, I am not sure I understand it. I will mull over it a bit and reply if I get it...) – 2011-11-12
-
0What puzzles me is why the assumption that $\theta$ is rational was included. Just as a distraction, maybe? Which could have been strengthened by making $\theta$ an integer, or even $0$ or even dropping the Gaussian assumption. Surely all continuous random variables $X$ are also irrational almost surely? – 2011-11-12
-
0@Dilip Yes, good point. It seems that all those details are merely distractions. – 2011-11-12
-
0@Srivatsan, Sorry for that, I will wait for more answers. Maybe that $\theta$ is rational is just to confuse us. – 2011-11-12
-
0What I want to ask is: for example, if we want to maximize the likelihood of $\ell(\theta; X)$ over the parameter space $\theta\in \mathbb{Q}$, then we'd like to find an mle $\hat\theta\in\mathbb{Q}$. But since $\bar X$ is a.s. irrational, it won't satisfy now. I feel I am getting confused. $\bar X$ is always an estimator for $\theta$ and it's almost surely irrational provided its distribution is continuous. However, if we restrict $\theta$ to some narrower space, say integer, then, $\bar X$ won't be the mle? Is that right? – 2011-11-12
-
0Zoe: your last question is a good entry point to what @Srivatsan explained (which is perfectly correct). For almost every $X$, one cannot maximize $\ell(\theta,X)$ over $\theta\in\mathbb Q$ because there is no $\theta\in\mathbb Q$ such that $\ell(\theta,X)\geqslant\ell(u,X)$ for every $u\in\mathbb Q$. Imagine for a moment that $X$ is uniform on $(0,1)$ and that $\ell(\theta,X)=1-(X-\theta)^2$. Obviously the best $\theta$ would be $\theta=X$ .../... – 2011-11-12
-
0.../... but $X$ is almost surely irrational so what can you do? For every irrational $x$ and every rational $u$, there exists another rational $v$ such that $\ell(v,x)<\ell(u,x)$. So, if you propose $u$ as a MLE of $\theta$, for **any** $u$, I can reply some $v$ which is strictly better than your proposal... That is, as soon as $X$ is irrational, but this happens with full probability. Take-home message: MLE restricted to non-closed sets of parameters is a **BAD** idea. – 2011-11-12
-
0@Sri, +1 for ignoring the irrelevant aspects of the question. – 2011-11-12