The elements of the Lie algebra can be thought of as "infinitesimal elements," so it's natural to think of them as being infinitesimally close to the identity; if we denote by $\epsilon$ an infinitesimal that squares to zero, then you can informally think of the Lie algebra as the set of elements of the form $I + \epsilon X$ where $I$ is the identity. For example, the elements of the orthogonal group satisfy $A \cdot A^T = I$, so the infinitesimal elements of the orthogonal group satisfy $(I + \epsilon X)(I + \epsilon X^T) = I + \epsilon (X + X^T) = 0$, or $X + X^T = 0$. Hence the Lie algebra of the orthogonal group is precisely the space of skew-symmetric matrices. (This argument can be made completely formal if we work with algebraic groups instead of Lie groups.)
There is a precise sense in which the Lie algebra consists of "infinitesimal elements": whenever a Lie group $G$ acts on a manifold $M$ by symmetries, the Lie algebra $\mathfrak{g}$ acts by differential operators on the smooth functions $M \to \mathbb{R}$. So they "infinitesimally generate" symmetries of $M$ (and the precise sense in which this is true is the exponential map).
If you prefer, there is an equivalent definition which does not privilege the identity element: the elements of the Lie algebra are the left-invariant vector fields on the Lie group. (Since a Lie group acts transitively on itself, a left-invariant vector field is determined by its value at any group element, and again, it's natural to look at the identity element.)