2
$\begingroup$

I've been trying to use the cauchy-schwarz inequality to prove that $(cov(X,Y))^2 \leq var(x)var(Y)$. The cauchy-schwarz inequality can be expressed as follows: if $u$ and $v$ are vectors in an inner product space then $^2 \leq (\parallel u\parallel)^2 (\parallel v\parallel)^2$. How do you define the vectors and the inner product so to prove the result in the first sentence. I've seen something like $(cov(X,Y))^2 = < (x-E(X)),(y - E(Y)) > \leq (\parallel (x-E(X) \parallel )^2(\parallel (y-E(Y) \parallel )^2 = var(X)var(Y) $ but how is $< (x-E(X)),(y - E(Y)) >$ expressed as a sum? Also how can you let $(x-E(X))$ and $(y - E(Y))$ be vectors?

Any insight would be great.

2 Answers 2

4

The inner product space you might have in mind is $L^2(\Omega, \mathscr A, P)$, the space of all square integrable random variables $X \colon \Omega \to \mathbb R$. Square integrable means that we have \[ \|X\|^2 := E(|X|^2) = \int_{\Omega} |X|^2\, dP < \infty \] The inner product is given by \[ \left := \int_\Omega XY\, dP. \] So it isn't a sum, but an integral and the vectors are the elements of the vector space $L^2(\Omega)$, that is, random variables.

Cauchy-Schwarz reads \[ \int_\Omega XY\, dP = \left \le \|X\|^2 \|Y\|^2 = \int_\Omega X^2\, dP \cdot \int_\Omega Y^2\, dP \] and applying this to $X-E(X)$ and $Y - E(Y)$ gives the desired inequality.


In cases where $P$ is discrete, that is there is a countable subset $\Omega'$ of $\Omega$ with $P(\Omega') = 1$, the integrals are sums, in this case we have \[ \left = \int_\Omega XY\, dP = \sum_{\omega \in \Omega'} X(\omega)Y(\omega) \] and \[ \left\|X\right\|^2 = \sum_{\omega \in \Omega'} X(\omega)^2. \]


$\def\cov{\operatorname{cov}}$ Addendum after comment: As you write, for discrete random variables we have \[ \cov(X,Y) = \sum_{\omega\in \Omega} \bigl( X(\omega) - E(X)\bigr)\bigl(Y(\omega) - E(Y)\bigr) \] If we want to write this as a sum over the values of $X$ and $Y$, just note that the joint probability mass (or as you write density) function $p_{X,Y}$ is given by \[ p_{X,Y}(x,y) = P({\omega \mid X(\omega) = x, Y(\omega) = y}) \] Grouping these terms in the above sum, we obtain \[ \cov(X,Y) = \sum_{x\in X[\Omega]}\sum_{y\in Y[\Omega]} \bigl(x- E(X)\bigr)\bigl(y-E(Y)\bigr)p_{X,Y}(x,y). \]

  • 0
    great, thanks again2012-10-11
0

The cauchy-schwarz inequality is for discrete case, (∑ (a_i)^2 ) (∑ (b_i)^2 ) ≥ ( ∑ (a_i)(b_i) ) ^2 , where the sum is over the suffix i from 1 to n ( n is the number of values under consideration). So, denoting the mean of the values of x by u and the mean of the values of y by t , we get by writing a_i = x_i - u and b_i = y_i - t , ( ∑ ( x_i - u)^2 )( ∑ ( y_i - t)^2 ) ≥ ( ∑ ( x_i - u)(y_i - t) )^2 , i.e. , by definition, { n.var(x) } { n.var(y)} ≥ {n.cov(x,y)}^2 , so that var(x) var(y) ≥ {cov(x,y)}^2 ; this is the proof of the inequality in case of Bivariate Analysis you may also extend it for probability distribution.

  • 0
    Sorry for late comment. You haven't used prob. density functions in your notation for variance and covariance but I think these are equivalent. I think you would need a double sum if there was a different no. of $y_{i}$s say with $x_{j}$s.2012-10-10