Abstract algebra, IMHO -- by a wide margin.
There are a wealth of applications of algebra to theoretical and even practical computer science. It's also beginning to go the other way -- theoretical computer science influencing research in abstract algebra.
Some example of the former (applications of algebra to CS) include the theory of groups, semigroups and monoids to automata and formal languages, particularly finite-state automata and languages. In fact, semigroup/monoid theory provides a very powerful and elegant way to look at finite automata -- for example a "recognizable" (finite state) set is $h^{-1}(P)$ where $P \subseteq M, M$ a finite monoid, $h:\Sigma^* \rightarrow M$ a monoid morphism, and a large, rich theory takes off from there.
In the other direction, a group is said to be "automatic" if it is finitely generated (it is all the products of a finite set of generator elements), group multiplication can be represented by a finite-state automaton, as can the property that a particular product of generators multiplies out to the groups unit. That leads to interesting application to groups in Euclidean space and on 3-manifolds and others.
Number theory is a wonderful branch of mathematics, perhaps the "purest" branch. It finds occasional application within theoretical CS, but not nearly to the same extent as algebra.
I wish I had taken more algebra courses as a CS grad student -- I'd have a lot less to catch up on now. More generally, the kind of thinking in abstract algebra -- axiomatization, morphism, composition, decomposition, quotient, etc etc will stand you in good stead throughout CS.