I don't think what you're asking for is possible, at least not for some rational primes. Let me build on the example Mr. Brooks gave in a comment yesterday.
The classic non-UFD is $\mathbb Z[\sqrt{-5}]$. Consider the rational prime $p = 3$. The norm function in the field $\mathbb Q(\sqrt{-5})$ is $N(a + b\sqrt{-5}) = a^2 + 5b^2$. Possible integer norms are 0, 1, 4, 5, 6, 9, ...
Since 3 is an impossible norm, that means 3 is irreducible. But 6, the first nontrivial positive multiple of 3, is a possible norm, and in fact $(1 - \sqrt{-5})(1 + \sqrt{-5}) = 6$.
So if the union of $\mathfrak P_1$ and $\mathfrak P_2$ contains all numbers in this domain with norm divisible by 3, it looks something like $\langle 3, 1 - \sqrt{-5} \rangle \cup \langle 3, 1 + \sqrt{-5} \rangle$.
But neither of those two ideals are principal. Or at least they don't look principal. If either of them was principal, we'd be able to solve $x \pm x\sqrt{-5} = 3$ or $3x = 1 \pm \sqrt{-5}$ with $x \in \mathbb Z[\sqrt{-5}]$.
What would be the norm of this number $x$? Remember that the norm is a multiplicative function. To solve the former equation, we would have to have $$N(x) = \frac{3}{2},$$ which is not an integer at all, and to solve the latter equation we'd need $N(x) = 2$, an impossible norm in this domain.
Now let's look at $2 + \sqrt{-5}$. Since this number has a norm of 9, it must be contained in $\mathfrak P_1 \cup \mathfrak P_2$. I emphasize that $(2 + \sqrt{-5}) \not\in \langle 3 \rangle$ nor $\langle 1 - \sqrt{-5} \rangle$ nor $\langle 1 + \sqrt{-5} \rangle$.
If $\mathfrak P_1 = \langle 3, 1 - \sqrt{-5} \rangle$, it consists of all numbers in this domain of the form $3\alpha + (1 - \sqrt{-5})\beta$, with $\alpha$ and $\beta$ also numbers in this domain. And since $3 + (-1 + \sqrt{-5}) = 2 + \sqrt{-5}$, that means $\alpha = 1$ and $\beta = -1$ is our verification that $(2 + \sqrt{-5}) \in \mathfrak P_1$.
EDIT: Thanks to Dave R. for pointing out a mistake of arithmetic. I've corrected it now.