$\def\e{{\bf e}} \def\D{\mathrm{diag}(\x)} \def\x{{\bf x}} \def\A{{\bf A}} \def\M{{\bf M}} \def\Mi{{\bf M}^{-1}} \def\P{{\bf P}_i} \def\id{\mathbb{I}} \def\tr{\mathrm{Tr}\,}$ Let $\M = \A + \D$. Note that $d(\M\Mi) = d \id = {\bf 0}$. Thus, $d\Mi = -\Mi d \M \Mi$, and so $\begin{eqnarray*} d \tr \Mi &=& d \sum \e_i^T \Mi \e_i \\ &=& \sum \e_i^T d \Mi \e_i \\ &=& -\sum \e_i^T \Mi d \M \Mi \e_i. \end{eqnarray*}$ If $\A$ is a function of $\x$, this is about as far as we'll go, $\begin{eqnarray*} \frac{\partial}{\partial x_i} \tr \Mi &=& -\sum_j \e_j^T \Mi \frac{\partial \M}{\partial x_i} \Mi \e_j \\ &=& -\tr \left((\Mi)^2 \frac{\partial \M}{\partial x_i}\right). \end{eqnarray*}$
If $\A$ is not a function of $\x$ we have that $\P = \frac{\partial \M}{\partial x_i}$ is a projection operator. (All components of $\P$ are zero except the $ii$th component, which is $1$.) In that case we find $\begin{eqnarray*} \frac{\partial}{\partial x_i} \tr \Mi &=& -\sum_j \e_j^T \Mi \P \Mi \e_j \\ &=& -\tr \left((\Mi)^2 \P\right) \\ &=& -(\Mi)^2_{ii} \\ &=& -\sum_j \Mi_{ij}\Mi_{ji}. \end{eqnarray*}$
In terms of $\A$, $\x$, and $\e_i$, $\begin{eqnarray*} \frac{\partial}{\partial x_i} f(\x) &=& -\sum_j \e_j^T (\A + \D)^{-1} \frac{\partial \D}{\partial x_i} (\A + \D)^{-1} \e_j \\ &=& -\sum_j \e_j^T (\A + \D)^{-1} \P (\A + \D)^{-1} \e_j. \end{eqnarray*}$ As noted by @passerby51 we could find an expression for $(\A + \D)^{-1}$ in terms of $\A^{-1}$ and $\D^{-1}$, but we will stop here.
Addendum: We made no assumptions about the basis $\e_i$. If $\e_i$ is the natural basis the above implies $\begin{eqnarray*} \frac{\partial}{\partial x_i} f(\x) &=& -\e_i^T (\A + \D)^{-2} \e_i, \end{eqnarray*}$ which agrees with @passerby51's result. (Note that in this case $\P = \e_i \e_i^T$.)
By the Woodbury formula, $\begin{eqnarray*} (\A+\D)^{-1} &=& \A^{-1} - \A^{-1}(\A^{-1} + \D^{-1})^{-1} \A^{-1}. \end{eqnarray*}$ This may or may not be useful depending on the form of $\A$.