A curve $c: I\rightarrow M \subset \mathbb{R}^n$ may be defined to be a geodesic iff it is a critical point for the length functional (so you look at smooth variations $\phi:(-\varepsilon, \varepsilon) \times I \rightarrow M$ s.t. $\phi(0,t) = c(t)$ and calculate the first variation of the length, $d/dt|_{t=0} L(\phi(t,.)$ and search for $c$ such this derivative is zero for any variation). It turns out that this results in a rather unpleasant result (second order ODE for $c$) which is due to the fact that the length functional is invariant wrt to parametrizations of the curve.
It turns out that the energy functional $\frac{1}{2}\int |c'|^2 dt$ has (geometrically) the same critical points as the length functional, but a critical point of the energy functional is necessarily parametrized proportionally to arc length. For these the property you mention (the orthogonal projection of $c^{''}(t) $ onto $T_{c(t)}M$ vanishes) in fact characterizes the geodesics in $M$ for submanifolds $M$ of Euclidean space when viewed as isometrically embedded submanifolds.
The equation satisfied locally for geodesics is also well known explicitly in terms of geometric data of the embedding $i:M\rightarrow \mathbb{R}^n$ and is, e.g. in a local coordinate representation of the form $\frac{d^2 c^k}{dt} + \sum_{ij}\Gamma^k_{ij}(c(t))c_i'(t) c_j'(t) = 0 \,\,\,\, (\forall k) $ That is, it is linear in the second derivatives and quadratic in the first derivatives but with coefficients which are nonlinear in $c$ itself. Here the $\Gamma^k_{ij}$ are the so called Christoffel symbols of the Manifold $M$ which are certain combinations of the first derivatives of the metric of $M$.
Most books on differential geometry will give detailed proofs of these facts and also have formulas for the Christoffel symbols.
(So, in general, checking whether a given curve is a geodesic may become a cumbersome computation. Your criterion is often helpful, e.g. to check that an equator is a geodesic on a sphere. For this reason one searches often for other means to find out that a curve is a geodesic, e.g. by looking for certain symmetries of the manifold).