Let $M(a,b)=\max\{ab,1-a-b+ab,a+b-2ab\}$, and observe that $1=ab+(1-a-b+ab)+(a+b-2ab)\leq 3M(a,b),$ so $\frac{1}{3}\leq M(a,b)$ for all such $a,b$. Hence, $\frac{1}{9}$ is ruled out. Can a value of $\frac{1}{3}$ actually be achieved? Well (as Cocopuffs pointed out), it can only be achieved if $\frac{1}{3}=ab=1-a-b+ab=a+b-2ab$, so let's see if that can happen.
Now, $ab=1-a-b+ab$ iff $1-a-b=0$ iff $a+b=1$. Hence, we'd need $a+b=1$ and $ab=\frac{1}{3},$ but this system yields $3a^2-3a+1=0$, which quadratic has a negative discriminant, and so no real solutions. Hence, there are no such real $a,b$, so $\frac{1}{3}$ will never be achieved, and we can rule it out as an answer.
If you try the special cases suggested by Mark Bennet, you'll see that it is possible for $M(a,b)=\frac{4}{9}$, so $\frac{5}{9}$ is ruled out as an answer. The only remaining question is: can we do better than $\frac{4}{9}$? Turns out that's the best we can do.
I really can't see a nice precalculus way to show that the answer is, in fact, $\frac{4}{9}$. Through methods of calculus, if we break $M(a,b)$ down piecewise, we can determine that it is minimized along the curves where the surface has a "fold", and then find that minimum explicitly. (See these W|A graphs for the idea. You'll want to click "Show contour lines" on the 3D plot.)