I do complex differential geometry, so for me these things pop up in the context of finite dimensional linear algebra. I'll address your question in those terms.
For references, I've profited much from Lee's Riemannian Manifolds and Coffman's Trace, metric and reality, which is an amazing book that talks about these things in a coordinate free way, which is almost essential to understanding what is going on.
So, let's take a finite dimensional real vector space $V$, and equip it with an inner product $g$. Here $g(u,v) = \langle u, v \rangle$ in your notation.
Each vector space $V$ is canonically isomorphic to its double dual $V^{**}$, but it is not to its dual $V^*$. Thus if we want to transport vectors from $V$ to $V^*$ (we want to do this a lot, just note that the transpose of a vector is an element of the dual space) we need to choose an isomorphism $V \to V^*$.
The inner product $g$ is exactly the choice of such an isomorphism. More precisely, given $u$ in $V$ we send it to the linear form $v \mapsto g(u,v)$. We could also have sent $u$ to the linear form $v \mapsto g(v,u)$, but as $g$ is symmetric this gives the same form.
Suppose now that we have a vector space of the form $V^{\otimes p} \otimes V^{*\otimes q}$. An element of this space is often called a $(p,q)$ tensor. Pick one of the spaces $V$, say the first one. Then the inner product $g$ induces an isomorphism $V^{\otimes p} \otimes V^{*\otimes q} \to V^{\otimes p-1} \otimes V^{*\otimes q+1}$. In the literature, this map is called "lowering the index", or, in musical notation, the "flat" map.
The classical "raising of index", or the "sharp" map, is now just the inverse of the isomorphism $V \to V^*$ given by $g$. As we are working over finite dimensional spaces, so that $V^{**} = V$, it coincides with the dual inner product $V^* \to V^{**} = V$.
This is far from being (only) abstract nonsense. As one example, this point of view clarifies immensly the definition of positive-definitiveness of a matrix $A$. Recall that a matrix $A$ (in the canonical basis of $\mathbb R^n$) is positive-definite if ${}^t u A u > 0$ for all non-zero vectors $u$. Recalling that we have the standard inner product on $\mathbb R^n$, we re-interpret this condition as saying that $\langle Au, u \rangle > 0$ for all vectors $u$.
But now it is immediatly clear that this property depends on the choice of an inner product, and not a basis. So, let $g$ be an inner product on $V$. Then we say that a linear morphism $f : V \to V$ is $g$-positive-definite if the bilinear form (i.e. map $V \to V^*$) $g \circ f$ is positive-definite, in the sense that $g \circ f (u,u) = g(f(u),u) > 0$ for all non-zero $u$. This does however not necessarily give the same condition as setting $g \circ f(u,u) = g(u,f(u))$.
To get around this, note that usually we only talk about positive-definiteness for symmetric matrices $A$. Such a matrix represents a linear map $f : V \to V$, or in other words, an element of $V^* \otimes V$. This last space is auto-dual, so $f^*$ is again a linear map $V \to V$. Saying that $A$ is symmetric exactly means that $f^* = f$. We are now free to define a $g$-positive-definite linear map $f : V \to V$ as a symmetric map as above.
As another example, this point of view will make clear what "trace with respect to a metric" actually means. (Hint: The trace of a linear map is invariant of basis, and has nothing to do with an inner product. A 2-form however, that is something else. If only there was a way to convert a 2-form into a linear map.)