Yes, you can certainly describe parabolae using linear algebra. The roots of a quadratic equation $\mathbf x^T\mathbf{Ax} + \mathbf b^T\mathbf x + c = 0$ in two dimensions lie on a conic curve, that is, a circle, ellipse, parabola, or hyperbola. The relationship is as follows:
When $\mathbf A$ has equal eigenvalues (i.e. it is a multiple of the identity), the curve is a circle.
When the eigenvalues have the same sign, the curve is an ellipse.
When one of the eigenvalues is zero, the curve is a parabola.
When the eigenvalues have opposite signs, the curve is a hyperbola.
Unfortunately, it doesn't quite work in the form you've used above; when you do $\mathbf x^T\mathbf{Ax} = 1$ with $\mathbf A$ having one vanishing eigenvalue, say $\mathbf A = \begin{bmatrix}1 & 0 \\ 0 & 0\end{bmatrix}$, what you get is a pair of parallel lines, $x = \pm1$. But if you add a linear term, say $\mathbf b^T\mathbf x$ with $\mathbf b = \begin{bmatrix}0 \\ 1\end{bmatrix}$, you do get a parabola, $x^2 + y = 1$. So maybe you can think of the two parallel lines as a sort of degenerate case of an infinitely stretched parabola.
This generalizes quite naturally to higher dimensions. The surfaces that result are called quadrics, and they include the familiar spheres, cylinders, and cones, as well as the less familiar but quite attractive ellipsoids, paraboloids, and hyperboloids. As you might expect, all of these can be classified using the eigenvalues of the corresponding matrix.