0
$\begingroup$

I am stuck to this code and particularly the point F=[ones(size(x)) x x.^2]. What does it do mathematically? I cannot understand what the heck the matrix has to do with this least-squares regression.

I think it should do something like:

$$\begin{align} \min \sum_{i}&r_{i}\\ s.t. \\ r_{i}=(o_{i}- &h_{i})^2 \end{align} $$

where $r_{i}$ is the distance between the observed value $o_{i}$ and the plotting value $h_{i}$.

I do not have access to MATLAB so please refer to some free software such as R or Python if you are going to demo.

x=[1 3 5 7 9 11]'
y=[0.0100 2.3984 11.0256 4.0272 0.2408 0.0200]'
plot(x,y,'r*')
hold on                                 %  Model a_1+a_2*x+a_3*x^2
F=[ones(size(x)) x x.^2]
a=F\y                                   % solves F*a=y with Least-Squares
l=max(x)-min(x);                        % greatest distance between x -points
t=[floor(min(x)):l/100:ceil(max(x))]';  % plot distance a bit greater
plot(t,[ones(size(t)) t t.^2]*a,'b')    % [ones(size(t)) t t.^2]*a calcs fits in t
legend('Data points','Fits')
axis([0,12,-2,15])
SSE=sum((y-F*a).^2)                     % a sum
SST=sum((y-mean(y)).^2)                 % total sum
rSquared=1-SSE/SST                      % R-squared
fun = @(x)(-a(1)-a(2)*x-a(3)*x^2)       % -1 * fit, cos fminsearch looks for min
x_max=fminsearch(fun,6)
plot(x_max,a(1)+a(2)*x_max+a(3)*x_max^2,'o')
hold off

...this code is supposed to do the same thing:

x = [1 3 5 7 9 11]’;
y = [0.0100 2.3984 11.0256 4.0272 0.2408 0.0200]’;
X = [x.^0 x x.^2];               % Model: y = b1 + b2x + b3x^2 -> y = Xb
x_grid = 0:0.01:11;              % grid for drawing
b1_hat = inv(X’*X)*X’*y;         % Least squares
y1_hat = b1_hat(1)*x_grid.^0 + b1_hat(2)*x_grid.^1 +...
b1_hat(3)*x_grid.^2;
figure(1)                        % Drawing the fitted polynomial
plot(x,y,’*’,x_grid,y_hat)
optimal1 = max(y_hat)

...I am now confused by this X = [x.^0 x x.^2];, the logic seems the same as earlier but I cannot understand yet it.

1 Answers 1

0

The reason why the rows of your design matrix take the form $(1\quad x_k\quad x_k^2)$ is that you're fitting with the quadratic model

$$y=c_0+c_1x+c_2x^2$$

If you expand out the matrix product

$$\begin{pmatrix}1&x_1&x_1^2\\1&x_2&x_2^2\\\vdots&\vdots&\vdots\\1&x_m&x_m^2\end{pmatrix}\begin{pmatrix}c_0\\c_1\\c_2\end{pmatrix}$$

you should be able to recognize that in the residual function you're trying to minimize.

In general, one always adds a column of $1$'s to the design matrix to account for constant terms in the model.

  • 0
    `b1_hat`, `y1_hat` and `b1_hat(3)`... why and what it really do? It does vector multiplication with the `X*X'` (where `X'` is apparently transpose(?!)) and then it takes inverse but why, sorry I cannot understand this yet...2011-12-03
  • 1
    Maybe [this answer on CV](http://stats.stackexchange.com/a/1882) might clarify a few things to you.2011-12-03