All of the examples you cite are examples of contravariant functors in action. In each case, we have a functor from manifolds to some other construction (all of the examples you cite are sections of some natural bundles you can associate to a manifold). Because this is a functor, this means that maps between manifolds give you maps between the the images of the manifolds under the functor.
Contravariance is a mechanism that arises naturally any time the functor you're considering does something along the lines of sending "X" to "a collection of things defined on X". This is because maps $X \rightarrow Y$ don't give you any way to turn a function/form on $X$ into a function/form on $Y$ (as the case with a covariant functor would be), but instead, by "pulling back" along the map, we can turn functions on $Y$ into functions on $X$.
To my mind, I understand pretty much all of the basic functors in differential geometry without diving heavily into categorical language this way: we can push forward subobjects along a map (so, for instance the tangent space map is covariant in light of its definition in terms of curves passing through a point, which can be pushed forward), but we have to pull back anything that is a map from the manifold to something else. The first place that I really understood the utility of the diagrammatical definition of "pullback" was in the context of pulling back vector bundles.
As for your question about pullback and adjoint both being notated with "*": The adjoint $T^*$ of a map $T$ in linear algebra is the image of $T$ under the contravariant functor $\operatorname{Hom}(\cdot,F)$, where $F$ is the ground field. It's pretty common notation to indicate the action of a contravariant functor by a superscript star. On a deeper categorical level, however, I'm not so sure. The "adjoint" concept generalizes to the notion of adjoint functors, while you've already seen the categorical take on the pullback. My experience with all of this is limited, however, so maybe someone else can give a better answer than this.