The homology of a closed manifold is a finitely generated abelian group, so we can write $H_i(M; \mathbb{Z}) \cong \mathbb{Z}^{b_i(M)} \oplus T_i,$ where $b_i(M)$ is the $i^\text{th}$ Betti number of $M$ and $T_i$ is the torsion part of $H_i(M;\mathbb{Z})$. Then we have that $\mathrm{Hom}(H_i(M;\mathbb{Z}), \mathbb{Z}) \cong \mathrm{Hom}(\mathbb{Z}^{b_i(M)},\mathbb{Z}) \oplus \mathrm{Hom}(T_i,\mathbb{Z}) \cong \mathbb{Z}^{b_i(M)}$ and $\mathrm{Ext}(H_i(M;\mathbb{Z}),\mathbb{Z}) \cong \mathrm{Ext}(\mathbb{Z}^{b_i(M)},\mathbb{Z}) \oplus \mathrm{Ext}(T_i,\mathbb{Z}) \cong T_i,$ where we have used the properties of $\mathrm{Ext}$ found at the bottom of page 195 in Hatcher.
Then the Universal Coefficient Theorem for cohomology says that $0 \longrightarrow \mathrm{Ext}(H_{i-1}(M;\mathbb{Z}),\mathbb{Z}) \longrightarrow H^i(M;\mathbb{Z}) \longrightarrow \mathrm{Hom}(H_i(M;\mathbb{Z}),\mathbb{Z}) \longrightarrow 0,$ or, by the above work, $0 \longrightarrow T_{i-1} \longrightarrow H^i(M;\mathbb{Z}) \longrightarrow \mathbb{Z}^{b_i(M)} \longrightarrow 0$ is (unnaturally) split exact. Hence $H^i(M;\mathbb{Z}) \cong \mathbb{Z}^{b_i(M)} \oplus T_{i-1}.$
Now in particular, since $M$ is a closed, orientable $2k$-dimensional manifold, by Poincaré duality, $H^k(M;\mathbb{Z}) \cong H_k(M;\mathbb{Z}),$ so from the above it must be that $T_{k-1} \cong T_k.$ So if $H_{k-1}(M;\mathbb{Z})$ is torsion-free, i.e. $T_{k-1} \cong 0$, then $H_k(M;\mathbb{Z})$ is torsion-free as well.