I'd love your help with this following problem:
Given $y_1,y_2,\ldots,y_N \in \mathbb{R}$, I need to prove that $f(x)= \sum \limits_{i=1}^{N}|x-y_i|^2$ gets a minimum, and to find it.
I don't know really how to solve it,
Any suggestions?
Thanks!
I'd love your help with this following problem:
Given $y_1,y_2,\ldots,y_N \in \mathbb{R}$, I need to prove that $f(x)= \sum \limits_{i=1}^{N}|x-y_i|^2$ gets a minimum, and to find it.
I don't know really how to solve it,
Any suggestions?
Thanks!
You can use calculus, or not. Expand the squares. We get $f(x)=Nx^2-2x\sum_{i=1}^N y_i+\sum_{i=1}^N y_i^2.$ This is a quadratic in $x$, and the coefficient of $x^2$ is positive. So the curve $y=f(x)$ is an upward facing parabola.
Thus $f(x)$ has a minimum, where the parabola is lowest, and it can be found in the usual calculus way.
But I would like to do it another way. Multiply $f(x)$ by $N$. We get $Nf(x)=N^2x^2-2xN\sum y_i + N\sum y_i^2.$ Complete the square. We get $Nf(x)=\left(Nx-\sum y_i\right)^2+N\sum y_i^2-\left(\sum y_i\right)^2.$ Then $Nf(x)$ is obviously minimized when $Nx-\sum y_i=0$, that is, when $x=\frac{\sum y_i}{N},$ since $\left(Nx-\sum y_i\right)^2$ is always $\ge 0$. The minimum value of $Nf(x)$ is therefore $N\sum y_i^2-\left(\sum y_i\right)^2$. It follows that the minimum value of $f(x)$ is equal to $\sum y_i^2-\frac{1}{N}\left(\sum y_i\right)^2.$
Remark: This is not just an "exercise": it has great practical importance, in Statistics and elsewhere. I will change the notation to the one the problem should have used, and write $x_i$ instead of $y_i$. So we have $N$ data points (numbers) $x_1,x_2,\dots,x_N$, and would like to find the value of $x$ which is in some sense nearest to our collection of data points, which best fits them. For technical reasons, we want to minimize the sum of the squares of the deviations from $x$ to the $x_i$, the sum of the squares of the "errors." So we are minimizing $\sum_{i=1}^N(x-x_i)^2.$ The minimum value is reached, as we saw, when $x=\frac{\sum x_i}{N}$. This is the average (mean) of the $x_i$.
Actually, we usually want to minimize $\frac{f(x)}{N}$. By our calculation, the minimum value of this is $\frac{\sum x_i^2}{N}-\left(\frac{\sum x_i}{N}\right)^2,$ which is usually called the variance of the collection of data points $x_i$. The variance in a sense measures how far, on average, the $x_i$ depart from the mean.
Hint: For what $x$ does f'(x)=0?