I'm slowly making my way through Strogatz's Nonlinear Dynamics and Chaos. One connection I made today is that artificial neural networks emerge from dynamical systems theory. If the weights are $\mathbf{\beta}$ then $\dot{\beta} = f(\beta) = -\frac{\partial E}{\partial \beta}$. The goal is (1) to minimize variance, and (2) to solve for a steady state of the system.
Then I started thinking about the potential energy metaphor, in which $$\dot x(t) = -\frac{dV}{dx}$$
If this were a physical system it would obey conservation of energy. I'm wondering if based on the definitions there is a conservation of energy, and if not, how to mathematically account for the loss of potential energy?