Here goes my answer.
Let $H_1$ and $H_2$ be two Sylow $2$-Subgroups. Then, $|H_1|=|H_2|=32$. So, we see that $|H_1 \cap H_2| \le 16$.
Now Let the normalizer of their intersection be $N(H_1 \cap H_2)$. Notice that $|H_1 \cap H_2| \in \{2,4,8,16\}$. Now, I claim that it cannot be $2$ or $4$. To see this, look at $H_1H_2 \subset G$.
$|H_1H_2|=\dfrac{|H_1||H_2|}{|H_1 \cap H_2|} \le 160 \implies |H_1 \cap H_2| \ge 7$
Case 1:
Suppose $|H_1 \cap H_2|=16$, then, $H_1 \cap H_2$ is normal in both $H_1$ and $H_2$.
This means, $H_1 \cup H_2 \subseteq N(H_1 \cap H_2)$.
$\begin{align*} |N(H_1 \cap H_2)|& \ge|H_1 \cup H_2|=|H_1|+|H_2|-|H_1 \cap H_2|\\&\ge32+32-16=48\\|N(H_1\cap H_2)|&\ge48\end{align*}$
But, $N(H_1 \cap H_2)$ is a subgroup of the group of order $160$ and by Lagrange's theorem, we have, $|N(H_1 \cap H_2)| \mid 160$
Again, the only divisors of $160$ greater than 48 are $80$ and $160$.
If $|N(H_1 \cap H_2)|=80$, then $N(H_1 \cap H_2)$ is a index 2 subgroup and hence normal.
If $|N(H_1 \cap H_2)|=160$, then $H_1 \cap H_2$ is itself normal.
*Alternatively, @mk points out, this case can be solved even easily, since, the cardinality of the normalizer is bigger than $2^5$,(as it contains $H_1$ as its subgroup) and, as $2^5$ divides it, we must have that the normalizer is the whole group! *
Case 2: $|H_1 \cap H_2|=8$
We have that $n_5=2^4$.
We claim that all Sylow $2$-subgroups must intersect in a group of order $8$.
Suppose there were $3$ distinct Sylow $2$-Subgroups, they will exhaust $93$ non-trivial elements leaving no room for the remaining $2$ subgroups.
Suppose there were $2$ distinct Sylow $2$-Subgroups, and even if remaining three subgroups intersect in a group of order $8$, they are going to have too many elements.($24 \times 3+7+31 \times 2$=$141$ non-identity elements)
Suppose there is a unique distinct group, even then there will be excess of elements.$24\times 4 +7+31=134$ non-identity elements.
Now, the only possibility we left open will also lead to a contradiction.
Because, in this case we require, $127$ elements.
So, a simple counting helps!