Consider the field $\mathbb{R}(x)$ of all (formal) rational functions in one variable with real coefficients. While this is not an ordered field, it is an orderable field -- it is possible to define an ordering $<$ on rational functions that is consistent with the usual laws of arithmetic.
For any ordering $<$ of $\mathbb{R}(x)$, we can define sets $L = \{ a \in \mathbb{R} \mid a < x\}$ and $R = \{ a \in \mathbb{R} \mid a > x\}$, and we have $\mathbb{R} = L \cup R$ -- under this ordering, every real number is either less than or greater than the polynomial $x$.
It turns out the ordering $<$ is completely determined by $L$ and $R$, and conversely each way to choose $L$ and $R$ corresponds to an ordering of $\mathbb{R}(x)$.
The complete list of orderings are:
- The ordering "$+\infty$" - $x$ is larger than every real number
- The ordering "$-\infty$" - $x$ is smaller than every real number
- The ordering "$a^+$" - $x$ is infinitesimally larger than $a$
- The ordering "$a^-$" - $x$ is infinitesimally smaller than $a$
The labels I've chosen for the orderings refer to "where" $x$ is placed in relation to the real line.
Some good buzzwords that relate to this sort of topic are:
- Real closed field
- Formally real field
- Real algebraic geometry
- Semi-algebraic geometry
There is an easy way to write down a first-order theory whose models are examples of the sort of number system you ask for. For example,
- Start with the language of ordered fields
- Add a new constant symbol $\varepsilon$
- Add in all of the ordered field axioms
- Add in one axiom $0 < \varepsilon$
- For every positive integer $n$, add in one axiom $\varepsilon < n$
Every model of this theory will have a number $\varepsilon$ with the property that it is larger than every non-positive real number, and smaller than every positive real number.