The following steps lead to a solution. Please note that I use the following notation: if $z,w\in G$, then $z^w=w^{-1}zw$, i.e., $z^w$ is the $\textit{conjugation}$ of $z$ by $w$. Similarly, if $H$ is a subgroup of $G$, then $H^w=\{z^w:z\in H\}$ is the $\textit{conjugation}$ of $H$ by $w$.
(1) Let $C=\textbf{C}_G(P)$ and $N=\textbf{N}_G(P)$ denote the centralizer and normalizer of $P$ in $G$ respectively. Also, let $x,y\in C$ be conjugate in $G$.
(2) The "trick" is to observe that $P\subseteq \textbf{C}_G(y)$ (the centralizer of $y$ in $G$) and $P\subseteq \textbf{C}_G(x)$. Choose $g\in G$ such that $y=x^g$. Prove that $(\textbf{C}_G(x^g))=(\textbf{C}_G(x))^g$.
(3) Deduce that $P^g\subseteq \textbf{C}_G(y)$. Hence $P$ and $P^g$ are Sylow $p$-subgroups of $\textbf{C}_G(y)$. Use the Sylow $C$-theorem in $\textbf{C}_G(y)$; i.e., use the fact that $P$ and $P^g$ are conjugate in $\textbf{C}_G(y)$.
(4) Conclude that $x$ and $y$ are conjugate in $\textbf{N}_G(P)$.
I believe this result is due to Burnside and is also referred to as \textbf{Burnside's Fusion Lemma}.