You don't get equation (6) from equations (4) and (5); those are only for determining what $x_1$ and $y_1$ are.
What you do is substitute $x(t)=x_1+\xi(t)$ and $y(t)=y_1+\eta(t)$ into the actual system of ODEs, which consists of equations (1) and (2) on p. 447. Then $\dot x(t)$ is replaced by $\dot \xi(t)$ (since $x_1$ is just a constant), $y+x-x^3/3$ is replaced by $(y_1+\eta)+(x_1+\xi)-(x_1+\xi)^3/3$, and so on.
After you've done that, and expanded everything on the right-hand side of the equations, you can simplify them by using that $x_1$ and $y_1$ are solutions of (4) and (5). This is important, since it will make all the constant terms cancel out, leaving only terms which contain powers of $\xi$ and/or $\eta$. And then, to get a linear system, one omits all terms of degree greater than one.
All this is a standard procedure called “linearization at an equilibrium point (or fixed point, or singular point)”, which is explained in any textbook on dynamical systems, so if you're unfamiliar with how to do it or what it's useful for, it's perhaps a good idea to find a book where you can read more about it.