Suppose I have vectors $\mathrm u \in \mathbb R^m$ and $\mathrm v \in \mathbb R^n$. I form the $m \times n$ rank-$1$ matrix
$$\mathrm A := \mathrm u \mathrm v^{\top}$$
and tell you what $\mathrm A$ is. Can you reconstruct vectors $\mathrm u$ and $\mathrm v$ from matrix $\mathrm A$? You cannot, because
$$\left( \gamma \mathrm u \right) \left( \frac{1}{\gamma} \mathrm v^{\top} \right) = \mathrm A$$
for all $\gamma \neq 0$. What you can recover from $\mathrm A$ is two lines passing through the origin and on which $\mathrm u$ and $\mathrm v$ live. More information is needed to recover $\mathrm u$ and $\mathrm v$.
For instance, if I tell you what $\| \mathrm u \|_2$ is, then you can find where the line whose direction vector is $\mathrm u$ intersects the Euclidean sphere of radius $\| \mathrm u \|_2$ centered at the origin. You find two points. One of them is $\mathrm u$ and the other is $-\mathrm u$, but you cannot know which is which, unfortunately. Sign ambiguity cannot be eliminated. Note that $(-\mathrm u) (-\mathrm v^{\top}) = \mathrm A$.
To summarize, matrix-valued function $\mathrm F (\mathrm x, \mathrm y) := \mathrm x \mathrm y^{\top}$ is not injective and, thus, the inputs $\mathrm x$ and $\mathrm y$ cannot be recovered from the output.