There is a purely algebraic theory of solutions of differential equations which may be viewed as providing the sort of differential analog that you seek. A differential field is a field with a derivation, i.e. linear map $\rm\:y\to y'$ satisfying the product rule $\rm\:(uv)' = u'v + v'u.\:$ Let $\rm\:F\:$ be a differential field of characteristic $0$, e.g. $\rm\,\Bbb C(x),\:$ and let $\rm\:L(y) = y^{(n)} + a_{n-1} y^{(n-1)}+\,\cdots\, + a_1 y' + a_0 y = 0,\:$ $\rm\, a_i\in F,\:$ be an $\rm\,n$-th order linear differential equation with coefficients in $\rm\,F.\,$ One can prove that there exists a minimal differential "splitting field" $\rm\,K\,$ containing all of the solutions of $\rm\:L(y)= 0.\:$ Indeed, $\rm\,K\,$ may be constructed from $\rm\,F\,$ simply by formally adjoining $\rm\,n\,$ $\rm\,F$-linear independent solutions $\rm\,y_1,\ldots,y_n$ of $\rm\,L(y)= 0,\,$ along with all their derivatives, yielding the differential extension field
$\rm K\, =\, F(y_1,\ldots,y_n, y_1',\ldots, y_n',\ldots, y_1^{(n-1)},\ldots,y_n^{(n-1)})$
Note that this field is closed under differentiation since the equation $\rm\:L(y) = 0\:$ allows us to rewrite all $\rm\:y_i^{(k)},\,\ k\ge n,\:$ in terms of lower order derivatives. This field is unique up to a differential $\rm\,F$-isomorphism and is known as the Picard-Vessoit extension of $\rm\,F\,$ associated to $\rm\:L.\:$ (Note: for simplicity, I omit some technicalities, e.g. preservation of constant subfields).
Further, there is a beautiful differential analog of Galois theory which, e.g. allows one to characterize algebraically the equations solvable in a ("louivillian") differential field obtained by successive adjunctions of exponentials, integrals or algebraic elements.
This purely algebraic theory is the foundation of constructive algorithms employed in computer algebra systems for symbolic manipulation of many common elementary functions.
For an introduction to these ideas you might find it enlightening to skim the introductory sections of Michael Singer's papers, and peruse his surveys, e.g. Formal solutions of differential equations, and An outline of differential Galois theory, and Introduction to Galois theory of linear differential equations.