I find it easier to think geometrically, so I'll argue that way. Suppose that $B$ is reduced and Noetherian, so that Spec $B$ is the union of finitely many irreducible components Spec $B_i$, each of which is reduced and irreducible, i.e. integral. Suppose also that each component of Spec $B$ dominates Spec $A$. (In ring-theoretic terms, suppose that each of the induced maps $A \to B_i$ is injective.)
Each of the maps $f_i:$ Spec $B_i \to$ Spec $A$ is also finite, because it is the composite of the closed immersion Spec $B_i \hookrightarrow $ Spec $B$ and the map $f:$ Spec $B \to $ Spec $A$, which is finite by assumption. Thus going down holds for each of the maps Spec $B_i \to$ Spec $A$.
Now any point $x$ of Spec $B$ belongs to one of the Spec $B_i$, and so given $y' \in $ Spec $A$ generalizing $y = f_i(x)$, going down gives $x' \in$ Spec $B_i$ generalizing $x$ such that $y' = f_i(x')$. Now think of $y$ just as an element of Spec $B$, and recall that $f_i$ is nothing but the restriction of $f$ to Spec $B_i$. We thus see that $f(y') = x'$, and so going down holds for the map $f$.
(You can easily convert this argument into pure commutative algebra: the point is that $B$ embeds into the product $\prod_i B_i$ of the domains $B_i$, and so any prime ideal of $B$ is pulled back from a prime ideal of one of the $B_i$. so going down for the $B_i$ implies going down for $B$. I leave the details to you.)
Note that it is crucial to assume that each Spec $B_i$ dominates Spec $A$, not just that Spec $B$ dominates Spec $A$. Otherwise we could just take Spec $B$ to be the disjoint union of Spec $B_1$ dominating Spec $A$, and some Spec $B_2$ which does not dominate Spec $A$; since going down does not hold for the map Spec $B_2 \to $ Spec $A$, it won't hold for the disoint union. This is what happens in wxu's counterexample: it is the disjoint union of a line and a plane mapping to a plane.