2
$\begingroup$

So here's the question:

Given a collection of points $(x_1,y_1), (x_2,y_2),\ldots,(x_n,y_n)$, let $x=(x_1,x_2,\ldots,x_n)^T$, $y=(y_1,y_2,\ldots,y_n)^T$, $\bar{x}=\frac{1}{n} \sum\limits_{i=1}^n x_i$, $\bar{y}=\frac{1}{n} \sum\limits_{i=1}^n y_i$.
Let $y=c_0+c_1x$ be the linear function that gives the best least squares fit to the points. Show that if $\bar{x}=0$, then $c_0=\bar{y}$ and $c_1=\frac{x^Ty}{x^Tx}$.

I've managed to do all the problems in this least squares chapter but this one has me completely and utterly stumped. I'm not entirely sure what the question is even telling me in terms of information nor do I get what it's asking. Any ideas on where to start?

  • 0
    @TongZhang Don't. Leave the post as it is, unless something is mathematically wrong in it.2013-04-01

2 Answers 2

1

You want to minimize:

$||\hat{y}-y||^2 = ||(c_0+c_1x)-y||^2$

where $\hat{y}$ are given by the linear regression. Intuitively, the best line minimizes the squared error between the "observed" ys and the "predicted" ys.

So, you need to find that $c_0,c_1$ that minimize this expression and show they are equal to those specified above.

Can you go on from there?

  • 0
    Unfortunately, no. $\hat{y}$ and $y$ are vectors, while $\bar{y}$ is a scalar (number) which is actually just the mean of the vector $y$. $\hat{y}$ is the vector created by taking the vector $x$ and multiplying each coordinate by a number ($c_1$) and then adding to each coordinate a number ($c_0$). In other words, these are the values "predicted" by the regression line for $x$.2012-10-14
1

The OLS estimators for $c_0$ and $c_1$ under the general setting(with no $\bar x=0$) are: $ c_0=\bar y- c_1\bar x, c_1= \frac{(x-\bar x\boldsymbol{1})^T(y-\bar y\boldsymbol{1})}{(x-\bar x\boldsymbol{1})^T(x-\bar x\boldsymbol{1})}, $ where $\boldsymbol{1}=(1,...,1)^T$ is a vector of one's and of length $n$. With the additional assumption $\bar x=0$, this reduces to $ c_0=\bar y, c_1= \frac{x^T(y-\bar y\boldsymbol{1})}{x^Tx} $