To elaborate on Pete L. Clark's answer: a lot of results stated in algebraic number theory books for complete valued fields (with, say, a discrete valuation) work just as fine for henselian fields (such as the algebraic numbers in $\mathbb{Q}_p$).
Why is this? In this particular example, the claim is that if $K$ is a henselian field (w/ a discrete valuation, say) and $L$ a finite separable extension, then the valuation on $K$ extends uniquely to $L$. The usual proof of this fact in the complete case -- basically, some argument based on the equivalence of all norms on a finite-dimensional space over a complete field -- doesn't (unless I'm missing something?) work here without completeness hypotheses.
But the result is still true. Namely, the point is that if $O$ is the ring of integers in $K$, then $O$ is a henselian ring. One way to state this is that Hensel's lemma holds for $O$, but a more transparent one is that any finite $O$-algebra splits as a direct product of local $O$-algebras. (You might find helpful Raynaud's book "Anneaux locaux henseliens," or sec. 4 of http://people.fas.harvard.edu/~amathew/chcompletion.pdf, here for the equivalence of the two claims.)
Now if O' is the integral closure of $O$ in $L$, then we know $O$ is a finite O'-module, so in particular it splits as a product of local rings. However, it's a domain! It follows that O' is itself local, and consequently a discrete valuation ring itself. In other words, the maximal ideal of $O$ extends in precisely one way to O' (this is equivalent to O' being local). But these extensions of maximal ideals correspond precisely to the extensions of valuations to $L$, so we get the claim.
The moral of the story is that much true for complete valued fields is true for henselian fields, because this (the uniqueness of extensions) is a key property of complete fields for statements in algebraic number theory.