Yes, the answer is "always".
(1) Let $f\in\mathbb{Q}[x]$ be an irreducible polynomial, let $F$ be some extension field of $\mathbb{Q}$ and assume that $f$ becomes reducible over $F$. Then $F$ contains an element $a\not\in\mathbb{Q}$ that is algebraic over $\mathbb{Q}$.
Proof: the coefficients of the irreducible factors of $f$ over $F$ are algebraic over $\mathbb{Q}$ and at least one of them is not in $\mathbb{Q}$.
(2) The rational function field $\mathbb{Q}(v)$ contains no elements $a\not\in\mathbb{Q}$ algebraic over $\mathbb{Q}$.
This is well-known and has already been discussed on this site several times.
(3) Let $a,b\not\in\mathbb{Q}$ be algebraic over $\mathbb{Q}$ and consider the field $F:=\mathbb{Q}(v,av+b)$, where $v$ is transcendental over $\mathbb{Q}$. Then $F$ contains elements $c\not\in\mathbb{Q}$ algebraic over $\mathbb{Q}$.
Proof: let $f\in\mathbb{Q}[x]$ be the minimal polynomial of a primitive element of the algebraic extension $\mathbb{Q}\subseteq\mathbb{Q}(a,b)=:K$. Then by (2) $f$ remains irreducible over $\mathbb{Q}(v)$. Hence $[K:\mathbb{Q}]=[K(v):\mathbb{Q}(v)]$.
By construction we have $FK=K(v)$, since $a=v^{-1}(av), b=v^{-1}(bv)$ in $FK$. Moreover $[FK:F]=[K(v):F]\leq [K(v):\mathbb{Q}(v)]$.
Assume now that $F$ contains no $c\not\in\mathbb{Q}$ algebraic over $\mathbb{Q}$. Then by (1) $[FK:F]=[K(v):\mathbb{Q}(v)]$.
Since degree of field extensions is multiplicative, we have the equation
$[FK:\mathbb{Q}(v)]=[FK:F][F:\mathbb{Q}(v)]=[FK:K(v)][K(v):\mathbb{Q}(v)]$
we then get $[F:\mathbb{Q}(v)]=[FK:K(v)]=[K(v):K(v)]=1$, hence the contradiction $a,b\in\mathbb{Q}$.