Here is my failed attempt. Please, see if it is far from being fixable or not.
Suppose $\mathcal{A}$ is a concrete category over $\mathcal{X}$, i.e. there is a faithful functor $U \colon \mathcal{A} \longrightarrow \mathcal{X}$ and for $X \in \mathcal{X}$, $\mathcal{F}^{X}$ is the following category:
Objects of $\mathcal{F}^{X}$ are pairs $\left(i,A\right)$, such that $i:X \longrightarrow A$ is a morphism! (1)
Morphisms of $\mathcal{F}^{X}$ are $f:\left(i_1,A_2\right) \longrightarrow \left(i_2,A_2\right)$ such that $f\circ i_1 = i_2$
Assuming that $U:$ Grp $ \longrightarrow$ Set is the underlying functor, the initial object of $\mathcal{F}^{X}$ for $X \in$ Set is the free group generated by $X$.
- Here is the catch, $i$'s are supposed to imitate the function from sets to the groups, in definition of free groups but obviously it is not clear where they live. I guess,they must be characterized in terms of $U$, the underlying functor.