The free objects you will encounter most often are free abelian groups (e.g $\mathbb Z^n$), free modules (e.g. vector spaces, say $\mathbb R^n$), free commutative algebras (polynomial rings, say $\mathbb C[X]$), free groups ($\mathbb Z$ again, on 1 generator, but on 2 and more generators you lose commutativity, elements are words in the generators).
The idea is building the simplest set with a desired algebraic structure from your chosen generators. For instance the simplest group from "$a$" is $\{a^n|\ n=...,-1,0,1,2,...\}$, which is abelian.
The wikipedia general definition of a free object is interesting but rather from the point of view of an algebraist, or category-theorist, it is fundamental in understanding algebraic theories as monads. It is really part of universal algebra, a subject with relatively few applications, though it is good to know it exists.
Also, I think there is an error in the wiki article, free objects are not analogues of bases of vector spaces, it is the generating sets of free objects that are.
Actually, from the wiki article: "let X be a set (called basis)" and later "A is the free object on X".
As Harald, I wonder why you mention calculus. It may be interesting to find contrived examples of free objects in calculus. First you would have to find categories in calculus.
EDIT: I should have stressed that the word "free" comes from "having no relation of dependence" -think "linear dependence" for example. This applies to the elements of your object. For instance, in a free abelian group you do not have any equality other than $xy=yx$ and those derived from this -$xyx=x^2y$, etc. (I cheat because you also have the associativity condition, $x(yz)=(xy)z$, the equality for 1, $1x=x$, and inverses, $xx^{-1}=1$, i.e. the group axioms.)