Suppose that $N$ students decide independently of each other to enroll in (i) class A only, (ii) class B only, (iii) both class A and class B, and (iv) neither class A nor class B, with probabilities $p$, $q$, $r$, and $s$ respectively where $p+q+r+s=1$. It is observed that $n_1-n_{12}$ students are enrolled in class A only, $n_2-n_{12}$ students are enrolled in class B only, and $n_{12}$ students are in both classes. As noted by Sasha, we have a multinomial distribution, and the likelihood of this observation is thus $q(N; n_1, n_2, n_{12}) = \frac{N!p^{n_1-n_{12}}q^{n_2-n_{12}}r^{n_{12}}s^{N-n_1-n_2 + n_{12}}}{(n_1-n_{12})!(n_2-n_{12})!n_{12}!(N-n_1-n_2 + n_{12})!}.$ To find the value of $N$ that maximizes $q(N; n_1, n_2, n_{12})$, we look at the ratio $\frac{q(N; n_1, n_2, n_{12})}{q(N-1; n_1, n_2, n_{12})} = \frac{Ns}{N-n_1-n_2+n_{12}}$ and note that the ratio is greater than $1$, (that is, q(N; n_1, n_2, n_{12}) > q(N-1; n_1, n_2, n_{12})), if $N < \frac{n_1+n_2-n_{12}}{1-s} = \frac{n_1+n_2-n_{12}}{p+q+r},$ and smaller than $1$, (that is, $q(N; n_1, n_2, n_{12}) < q(N-1; n_1, n_2, n_{12})$), if the above inequality is reversed. In other words, the maximum-likelihood estimate of $N$ is $\hat{N} = \frac{n_1+n_2-n_{12}}{p+q+r} = \frac{\text{total enrollment in classes A and B}}{P(\text{student enrolls in A or B or both})}$ where floors and ceilings have been ignored for simplicity of exposition. Of course, if we do not know $p+q+r$, this estimate is not very useful. Varying $p+q+r$ to find the maximum value of $\hat{N}$ is futile. It leads to an estimate of an infinite number of students none of whom take courses A or B. So much for higher education!