For this answer I will need the following 2 facts:
(1) Given a Borel subalgebra $\mathfrak{b}$ of a semisimple Lie algebra $\mathfrak{g}$, the nilradical $\mathfrak{n}^+$ of $\mathfrak{b}$ is the orthogonal complement of $\mathfrak{b}$ in $\mathfrak{g}$ with respect to the Killing form, and is moreover equal to the set of elements of $\mathfrak{b}$ that act ad-nilpotently on $\mathfrak{g}$.
(2) Given an element $x$ of a Lie algebra with an associative bilinear form (such as the Killing form), the $\mathrm{ad}(x)$ generalized eigenspaces for eigenvalues $\lambda$ and $\mu$ are orthogonal unless $\mu=-\lambda$.
Let $\mathfrak{b}$ be a Borel subalgebra of $\mathfrak{g}$ containing the nilradical $\mathfrak{n}=\mathfrak{rn(p)}$ of $\mathfrak{p}$. Our aim is to prove that $\mathfrak{b} \subseteq \mathfrak{p}$, since a parabolic subalgebra is by definition a subalgebra that contains a Borel subalgebra.
Choose $x \in \mathfrak{n}$. Since $\mathfrak{n}$ is a nilpotent ideal of $\mathfrak{p}$, the linear map $\mathrm{ad}(x)$ is nilpotent on $\mathfrak{p}$, and writing $\mathfrak{g}=E_0 \oplus E$ where $E_0$ is the $0$ generalized eigenspace and $E$ is the sum of the non-zero generalized eigenspaces of $\mathrm{ad}(x)$, we must have $E_0$ and $E$ orthogonal with respect to the Killing form (fact 2), so that $E$ is orthogonal to $\mathfrak{p}$. The latter contains its own orthogonal complement so $E$ is zero and it follows that $\mathrm{ad}(x)$ is nilpotent on $\mathfrak{g}$. Combining this with fact 1, $\mathfrak{n} \subseteq \mathfrak{n}^+=\mathfrak{b}^\perp$, implying that $b \subseteq \mathfrak{n}^\perp=(\mathfrak{p}^\perp)^\perp=\mathfrak{p}$.