Let's start with a well known limit: $$\lim_{n \to \infty} \left(1 + {1\over n}\right)^n=e$$
As $n$ approaches infinity, the expression evaluates Euler's number $e\approx 2.7182$. Why that number? What properties does the limit have that leads it to 2.71828$\dots$ and why is this number important? Where can we find $e$, in what branch of Mathematics?
As Walter Rudin puts it in his famous "Real and Complex Analysis":
This is the most important function in mathematics. It is defined, for every complex number $z$, by the formula
$$\exp(z)=\sum_{n=0}^{\infty}\frac{z^{n}}{n!}$$
There are alternative definitions and the "problem" of this one is that it requires notions about convergence of series, etc. But one big advantage - and it is why Rudin introduces it this way - is that it allow us to define diverse important functions directly from this one: $\sin$, $\cos$, $\log$, etc. let alone the famous Euler's identity. From the exponential function, we can also define the power of any positive number, which is not trivial (indeed, why would there be a number $3^{\sqrt{2}}$ for example?). It can also be used to define $\pi$.
It also appears that it is the only non-zero function that is equal to its derivative, which is a practical property in many situations.
In mathematics, one often wants to define things by making links with existing ones, but overall to define useful things for our purposes. The ubiquity of the exponential comes from the fact it appears to fulfill this very goal of being useful. The fact that it is equal to $2.7\dots$ has no real importance (what is important is that it is greater than $1$, say).
If you have some knowledge of calculus (convergence of series), the introduction of the Rudin can be a good read. Actually, even if you don't have any knowledge of series, it can be convincing that it is indeed "the most important function in mathematics". (Of course, there are other important functions and a "function" is a very general concept, but in analysis, topology, differential geometry, it is quite important).
I know only one amazing property.
For any number $a$, $(a^x)^\prime = ba^x$,where $\displaystyle b = \lim_{\Delta x \to 0} {a^{\Delta x} - 1\over \Delta x} = st\left({a^{\Delta x} - 1\over \Delta x}\right)$
eg :-
$$(10^x)^\prime = b(10^x)$$ Where $$b = \lim_{\Delta x \to 0} {10^{\Delta x} - 1\over \Delta x} = st\left({10^{\Delta x} - 1\over \Delta x}\right) = 2.303 \cdots$$
It just so happens that for only $e$, $b = 1$ so $(e^x)^\prime = e^x$
Which is just so amazing.
Firstly,
\begin{align*} e^{x} &= 1+x+\frac{x^2}{2!}+\frac{x^3}{3!}+\ldots \\ (e^{x})' &= e^{x} \end{align*}
Secondly,
\begin{align*} y &= e^{x} \\ \frac{dy}{dx} &= y\\ x &= \ln y \\ \frac{dx}{dy} &= \frac{1}{y} \\ \frac{d}{dy} \ln y &= \frac{1}{y} \\ \int \frac{dy}{y} &= \ln |y|+C \end{align*}
That's why we say $\log_{e} x=\ln x$ as natural logarithm.
Similarly, we use radian as a natural unit for angles (in calculus).
$e$ is used in recursive interest.
It's related to the fact that 5% interest ever 6 months is better than 10% every 12 months
Euler's number is the limit of intrest, ie: if you made an infinitesimal amount of interest constantly (at an infinitesimal interval, in the end, you would have $x*e$ money, x being the money you started with.
A popular forumula containing ie is: $e^{\pi i} = -1$
It is not so important that it equals $2.718\cdots$ but that it converges to some number.
When Napier,Jacob Bernoulli, et. al. were looking at it, they were thinking about the implications of the compound interest calculation. And the effect of increasing the compounding frequency has an upper bound.
Euler, a student of Bernoulli's, was working with the calculus of exponential functions and found $e^x$ to be unchanged when subjected to differentiation. This makes it an incredibly important function to the calculus.
He then went on to show how the exponential function becomes a trigonometric function when when taken to an "imaginary" power. That is $e^{ix} = \cos x + i\sin x$ or $e^{\pi i} = -1.$ This opens up the field of complex analysis.