1
$\begingroup$

I am taking a machine learning course and today we were given an example of regression, with two attributes $x_,x_2$ and $y$ being the real valued outcome.
$y$ is a quadratic function of $x_1,x_2$, given as
$$y=\alpha_1 x_1^2+\alpha_2 x_2^2+\alpha_3 x_1x_2+\alpha_4x_1+\alpha_5x_2+c$$ Now my teacher told that we can view this function as a linear function of $\lt x_1^2,x_2^2,x_1x_2,x_1,x_2 \gt$.
But I can't really visualize how can a curved function look like a straight line.
Please some one explain it or give some good reference.

  • 1
    It is a strange way to look at it. You can define the linear (actually affine) function $f(z_1,z_2,z_3,z_4,z_5) = c+\sum_{i=1}^5 \alpha_i z_i$, and then you have $y = f(x_1^2,x_2^2,x_1 x_2 , x_1, x_2)$, but its not clear why you want to do that.2012-07-27
  • 1
    Perhaps the best idea is not to think of it geometrically but algebraically. We need to pretend $x_1^2,x_2^2,x_1x_2,x_1,x_2$ are *five* independent variables, and I don't think attempting to visualize five-dimensional hyperplanes in relation to a quadratic surface in three dimensions is the way; just work formally. BTW, in what context are you supposed to view it as a linear function? The details of the specific application might lend to a more narrowly tailored explanation behind why such an interpretation works.2012-07-27
  • 0
    @anon It is related to support vector machines. Although he hadn't taught about SVMs but he just gave an idea that we can view every function as a straight line in higher dimensions.2012-07-27
  • 0
    I haven't time for a complete answer, but Figures 93 and 94 on [this page](http://www.cs.wustl.edu/~pless/546/lectures/L13.html) show how the simpler case of how $f(x,y) = ax + by + c(x^2+y^2)$ can be visualized as a linear function of $(x,y,x^2+y^2)$. The point is that a nonlinear function of the original data is turned into a linear function of a curved warping of the data.2012-07-27
  • 0
    http://en.wikipedia.org/wiki/Linear_combination2012-07-30

1 Answers 1