How can one prove that $\operatorname{Cov}(\cdot, \cdot)$ is symmetric and bilinear, where $(\cdot,\cdot)$ represents the dot product of two vectors?
I tried searching for it but couldn't find the answer. I need to prove that the covariance operator is symmetric & bilinear.
Show that $Cov(X,Y) = Cov(Y,X)$ -- symmetric
$Cov(aX,Y) = a Cov(X,Y)$ -- Linear
and if it is linear and symmetric it will be bi-linear.
$Cov(X,aY) = a Cov(X,Y)$ -- bi-linear
Now you can refer to the definition of co-variance, $E[(X- E[X])(Y-E[Y])]$
Now you can refer to the definition of co-variance, $E[(Y-E[Y])(X- E[X])]$ by the commutitivity of multiplication.
$E[X]$ is linear.
$E[aX] = a E[X]$
$Cov(aX,Y) = E[(aX- E[aX])(Y-E[Y])] = E[a(X- E[X])(Y-E[Y])] = aE[(X- E[X])(Y-E[Y])]$
Or you can say that Cov meets the definition of an inner product space, which must be symmetric (for vectors over the real numbers, conjugate symmetric for complex vector spaces). And linear (in the first term for complex vector spaces, bi-linear for real spaces).