First, let's be clear:
If $R$ is a commutative ring with $1$, then an element $a\in R$ is irreducible if and only if $a$ is not a unit, $a$ is not a zero divisor, and whenever $a=bc$ in $R$, either $b$ is a unit (invertible) or $c$ is a unit. Added. Elements are not usually called "reducible", though. Elements that are not irreducible are "not irreducible".
In the particular case of $R=F[x]$ with $F$ a field, this agrees with the "usual" notion of "irreducible polynomial", since the units are precisely the nonzero constant polynomials.
Now, if $F$ is a field, then $F[x]$ is a unique factorization domain (even more, a Euclidean domain). If $a(x)$ is irreducible, then it is a prime in $F[x]$. So for any $p(x)\in F[x]$ there are only three possibilities: either $a(x)$ divides $p(x)$, or $p(x)=0$, or else $a(x)$ and $p(x)$ are relatively prime, and in particular, there exist polynomials $r(x)$ and $s(x)$ such that $a(x)r(x) + p(x)s(x) = 1.$
Now, suppose that $p(x)$ is a polynomial. The only way for $a(x)$ to be a multiple of $p(x)$ is for $a(x)$ to be of the form $a(x)=\lambda p(x)$ for some nonzero constant $\lambda\in F$. Assume that $a(x)$ is not a multiple of $p(x)$.
If $p(x)\neq 0$ and $a(x)$ divides $p(x)$, then $p(x) = a(x)q(x)$ for some $q(x)\notin F$. We cannot have $p(x)$ dividing $q(x)$, so then $\overline{q(x)}$ is not zero in $F[x]/\langle p(x)\rangle$. Neither is $\overline{a(x)}$. Yet $\overline{a(x)}\times\overline{q(x)} = \overline{a(x)q(x)} = \overline{p(x)} = 0.$ So $\overline{a(x)}$ is not irreducible, because it is a zero divisor.
If $a(x)$ does not divide $p(x)$, then we can write $1 = a(x)r(x)+p(x)s(x)$ for some polynomials $r(x)$ and $s(x)$. But then in $F[x]/\langle p(x)\rangle$ we have: $1 = \overline{1} = \overline{a(x)r(x) + p(x)s(x)} = \overline{a(x)}\overline{r(x)} + \overline{p(x)}\overline{s(x)} = \overline{a(x)}\overline{r(x)},$ so $\overline{a(x)}$ is not irreducible because it is a unit.
Finally, if $p(x)=0$, then $F[x]/\langle p(x)\rangle$ is essentially just $F[x]$ itself again, so $a(x)$ is irreducible in the image because it's just $a(x)$ in $F[x]$.
So, to answer the question that was posed, if $p(x)\neq 0$ then $a(x)$ will not be irreducible in $F[x]/\langle p(x)\rangle$ no matter what.
That said...
But your use of the word "reducible" suggests that you are actually wondering about something else: whether "as a polynomial" it can be written as a product of two nonconstant polynomials, or perhaps whether "it has a root" when considered in $F[x]/\langle p(x)\rangle$.
In other words: say $p(x)$ is a nonconstant polynomial in $F[x]$. Then $R=F[x]/\langle p(x)\rangle$ is a ring that contains a copy of $F$. We can then consider the polynomial $a(x)$ as having coefficients in $R$ and ask whether it is reducible or irreducible over $R$. However, using $x$ here is confusing because $x$ is already playing a role in the definition of $R$, so it's better to switch to a different letter in order to talk about "polynomials with coefficients in $R$". So let's use $y$ to denote polynomials with coefficients in $R$, and so we will be asking whether $a(y)$ is irreducible over $R$ or reducible over $R$.
If $p(x)$ is irreducible over $F$, then $F[x]/\langle p(x)\rangle$ is a field that contains $F$, and you can consider $a(y)$ as a polynomial in $(F[x]/\langle p(x)\rangle[y]=R[y]$. There, it may be that $a(y)$ is irreducible or that it is reducible.
For an example where it is reducible, take $a(x) = x^4 + 1$ in $\mathbb{Q}[x]$, and $p(x) = x^2+1$, both irreducible. Since $\mathbb{Q}[x]/\langle p(x)\rangle = \mathbb{Q}(i)$, the rational complex numbers, in $\mathbb{Q}(i)[y]$ we have $a(y) = y^4+1 = (y^2+i)(y^2-i)$, reducible.
For an example where it is irreducible, take $a(x) = x^2 - 2$ and $p(x) = x^2-3$ in $\mathbb{Q}[x]$. Then $\mathbb{Q}[x]/\langle p(x)\rangle$ is just $\mathbb{Q}(\sqrt{3})$, and $a(y)$ has no roots in $\mathbb{Q}(\sqrt{3})$; being degree $2$, it is irreducible in $\mathbb{Q}(\sqrt{3})[y]$.
If $p(x)$ is reducible, then you end up with a ring that is not a field and has zero divisors. In any case, as a consequence of the Chinese Remainder Theorem, if we factor $p(x)$ into irreducibles, $p(x) = q_1(x)^{\alpha_1}\cdots q_m(x)^{\alpha_m},$ with $q_i$ and $q_j$ coprime for $i\neq j$, $\alpha_i\gt 0$, then $R=F[x]/\langle p(x)\rangle \cong \frac{F[x]}{\langle q_1(x)^{\alpha_1}\rangle}\times\cdots\times \frac{F[x]}{\langle q_m(x)^{\alpha_m}\rangle},$ and this $a(y)$ may or may not be irreducible over this ring.
For an example where it is reducible, take the example above in which $a(y)$ was reducible, but replace $p(x)$ with $(x^4+1)(x^2+2)$ or some other reducible polynomial that has $x^4+1$ as a factor. Then $a(y)$ will have a root in the quotient, because it will have a root in one of the factors. Actually, since $F$ is embedded into the quotient "diagonally", you need two irreducible polynomials over which $a(x)$ was reducible, or a power of one over which $a(x)$ was reducible. For an example where it is irreducible, do the same but now take the product of two polynomials over which $a(y)$ was irreducible. you can take $F=\mathbb{Q}$, $a(x) = x^2-2$, $p(x) = (x^2-3)(x^2-5)$. Then $F[x]/\langle p(x)\rangle \cong \mathbb{Q}(\sqrt{3})\times\mathbb{Q}(\sqrt{5})$, and that ring does not have a square root of $2$ either, so $a(y)$ is still irreducible.