Your problem is equivalent to find the transformation between the $x,y$ coordinates of a point and the $x^{\prime },y^{\prime }$ coordinates of the same point in a rotated system of coordinates, followed by a multiplication by the factor $k=1/\sqrt{x^{2}+y^{2}}$, so that $x^{\prime \prime }=kx^{\prime }=1$ and $y^{\prime \prime }=kx^{\prime }=0$. The rotation angle should be $\theta =\arctan \frac{y}{x}$.

From trigonometry, we know that
$ \begin{eqnarray*} &&\left\{ \begin{array}{c} x^{\prime }=x\cos \theta +y\sin \theta =\sqrt{x^{2}+y^{2}} \\ y^{\prime }=-x\sin \theta +y\cos \theta =0 \end{array} \right. \end{eqnarray*} $ and since
$ \begin{eqnarray*} \cos \left( \arctan \frac{y}{x}\right) &=&\frac{x}{\sqrt{x^{2}+y^{2}}} \\ \sin \left( \arctan \frac{y}{x}\right) &=&\frac{y}{\sqrt{x^{2}+y^{2}}}, \\ \end{eqnarray*} $
we have $\begin{eqnarray*} \left\{ \begin{array}{c} x^{\prime \prime }=\frac{1}{\sqrt{x^{2}+y^{2}}}x^{\prime }=\frac{x^{2}}{ x^{2}+y^{2}}+\frac{y^{2}}{x^{2}+y^{2}}=1 \\ y^{\prime \prime }=\frac{1}{\sqrt{x^{2}+y^{2}}}y^{\prime }=-\frac{xy}{ x^{2}+y^{2}}+\frac{xy}{x^{2}+y^{2}}=0. \end{array} \right. \end{eqnarray*} $
We haven't learned any matrices at school yet.
In matrix notation$^1$
$ \begin{eqnarray*} \begin{pmatrix} x^{\prime \prime } \\ y^{\prime \prime } \end{pmatrix} &=&\frac{1}{\sqrt{x^{2}+y^{2}}} \begin{pmatrix} x^{\prime } \\ y^{\prime } \end{pmatrix} = \begin{pmatrix} \frac{x}{x^{2}+y^{2}} & \frac{y}{x^{2}+y^{2}} \\ -\frac{y}{x^{2}+y^{2}} & \frac{x}{x^{2}+y^{2}} \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} 1 \\ 0 \end{pmatrix}. \end{eqnarray*} $
So $ M= \begin{pmatrix} \frac{x}{x^{2}+y^{2}} & \frac{y}{x^{2}+y^{2}} \\ -\frac{y}{x^{2}+y^{2}} & \frac{x}{x^{2}+y^{2}} \end{pmatrix}. $
--
$^1$ Product of a $2\times 2$ matrix by a $2\times 1$ matrix $ \begin{pmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{pmatrix} \begin{pmatrix} b_{1} \\ b_{2} \end{pmatrix} = \begin{pmatrix} a_{11}b_{1}+a_{12}b_{2} \\ a_{21}b_{1}+a_{22}b_{2} \end{pmatrix} $
and product between a scalar $\alpha$ and a $2\times 1$ matrix
$\alpha \begin{pmatrix} c_{1} \\ c_{2} \end{pmatrix} = \begin{pmatrix} \alpha c_{1} \\ \alpha c_{2} \end{pmatrix} $