If I'm reading this right, you want to know how gravity influences your sensor based on its rotation and then remove the effect of that influence. I think the best way to go about this is to rotate the gravity vector to correspond with your sensor's rotation and then add the vector that is opposite in magnitude to add out gravity. Like so:
Let $\vec{G} = \langle 0, 0, g \rangle$ where $g$ is the force of gravity, typically $-9.81 ^{m}/_{s}$.
The matrix for rotating a vector by pitch $\alpha$, yaw $\beta$, and roll $\gamma$ is: (from http://planning.cs.uiuc.edu/node102.html).
$$ \begin{equation} \begin{split} R(\alpha,& \beta,\gamma) = R_z(\alpha) \, R_y(\beta) \, R_x(\gamma) = \\ & \begin{pmatrix} \cos\alpha \cos\beta & \cos\alpha \sin\beta \sin\gamma - \sin\alpha \cos\gamma & \cos\alpha \sin\beta \cos\gamma + \sin\alpha \sin\gamma \\ \sin\alpha \cos\beta & \sin\alpha \sin\beta \sin\gamma + \cos\alpha \cos\gamma & \sin\alpha \sin\beta \cos\gamma - \cos\alpha \sin\gamma \\ -\sin\beta & \cos\beta \sin\gamma & \cos\beta \cos\gamma \\ \end{pmatrix}. \end{split} \end{equation} $$
Thus, the transformed gravity vector is:
$\vec{G}\,' = \vec{G}*R(\alpha, \beta,\gamma) = \langle -g \sin(\beta) , \;\; g \cos(\beta) \sin(\gamma) , \;\; g \cos(\beta) \cos(\gamma) \rangle$
You can then take your acceleration vector $\vec{A}$ and add the vector of opposite magnitude, which is the same as subtracting $\vec{G}\,'$. Thus, your final formula is:
$\vec{A}\,' = \vec{A} - \langle -g \sin(\beta) , \;\; g \cos(\beta) \sin(\gamma) , \;\; g \cos(\beta) \cos(\gamma) \rangle$
Is that what you want?
EDIT2: This matrix depends on the order in which rotations are applied.