1
$\begingroup$

Let $X_1,...,X_n$ be independent identically distributed random variables. I am trying to understand some of my statistics notes. My teacher states

The joint density of $Y=(X_{(1)},...,X_{(n)})$ where $X_{(i)}$ is the $i$'th order statistic of the $X_i$, is $f_{Y}(y)=n!f_{X}(x_{(1)})...f_{X}(x_{(n)})$ where $f_X$ is the p.d.f of $X$.

I do not understand what this particular formula means or where it came from. Any insight much appreciated. In particular I am confused where the $n!$ comes from.

1 Answers 1

2

The joint distribution is what your professor says but there's an important caveat: it only has support on the region where $x_{(1)}\le x_{(2)}\le \ldots \le x_{(n)}$ (which is obvious... it can never be the case that they're not in order).

As such, the $n!$ is a normalization constant. You know something like that must be there cause you aren't integrating $f_X(x_1) \ldots f_X(x_n)$ over the entire region.

As you might guess, its origin is combinatorial. There are $n!$ different possible orderings of the variables.

Let's start with the case where $n=2.$ The pdf for the order statistics has support on the half-plane $x_{(1)}\le x_{(2)}.$ We must look at how the variables $(X_1,X_2)$ map into the order statistics $(X_{(1)}, X_{(2)}).$ There are two options. We can have $X_1 < X_2$ in which case $X_{(1)}=X_1$ and $X_{(2)}= X_2$ or we can have $X_2 < X_1$ in which case $X_{(1)}=X_2$ and $X_{(2)}= X_1.$

So if we have $(X_{(1)},X_{(2)})$ near the point $(x_{(1)},x_{(2)}),$ (with $x_{(1)}\le x_{(2)}$), then that could be for one of two reasons. Either $X_1$ is near $x_{(1)}$ and $X_2$ is near $x_{(2)}$ or the other way around. Thus the PDF for the order statistics gets contribution from both of these regions of the distribution of $(X_1,X_2)$ and we have $$f_{X_{(1)},X_{(2)}}(x_{(1)},x_{(2)}) = f_{X_1,X_2}(x_{(1)},x_{(2)}) + f_{X_1,X_2}(x_{(2)},x_{(1)}) \\= f_X(x_{(1)})f_X(x_{(2)})+f_X(x_{(2)})f_X(x_{(1)}) \\= 2f_X(x_{(1)})f_X(x_{(2)})$$ where in the second line we used the fact that $X_1$ and $X_2$ are independent and identically distributed.

For $n>2$ it's the same thing only there are $n!$ contributions from the different orderings of $X_1\ldots X_n.$ Again, by symmetry each contribution adds the same amount to the density, so there's just an overall prefactor of $n!$.

More generally, if $X_1,\ldots, X_n$ are not i.i.d. and have joint PDF $f_{X_1,\ldots,X_n},$ then we have $$ f_{X_{(1)},\ldots, X_{(n)}}(x_{(1)},\ldots,x_{(n)}) = \sum_{\sigma\in S_n} f_{X_1,\ldots,X_n}(x_{(\sigma(1))},x_{(\sigma(2))},\ldots,x_{(\sigma(n))})$$ where $\sigma\in S_n$ are all the permutations of $\{1,2,\ldots, n\}.$

  • 0
    Great Answer. Two questions tho, what do you mean the region $x_{(1)} \leq ... x_{(n)}$ and what do you mean by "from $X_1 - X_2$ space". – 2017-02-12
  • 1
    For the first one, the region of $n$-dimensional space for which that inequality holds true. In the case of two variables it would be the region $x_{(1)} \le x_{(2)}$ in other words the half plane above the line $y=x$ if you plot $x_{(1)}$ on the $x$-axisand $x_{(2)}$ on the $y$-axis. – 2017-02-12
  • 1
    By $X_1-X_2$ space I mean that the order statistics $(X_{(1)},X_{(2)})$ are a function of $(X_1,X_2)$ and thus regions of two dimensional plane $(X_1,X_2)$ map under that function to regions of the aformentioned half plane. The point $(x_{(1)},x_{(2)})$ gets mapped to by two different points of the $(x_1,x_2)$ plane. One in the $x_2>x_1$ half-plane where $x_2 = x_{(2)}$ and $x_{1} = x_{(1)}$ and one from the $x_1>x_2$ half plane where $x_2= x_{(1)}$ and $x_1 = x_{(2)}$ – 2017-02-12