No. For example, take the ring of $2\times 2$ matrices over the even integers, $2\mathbb{Z}$. Let $J$ be the subring of all matrices of the form $\left(\begin{array}{cc} 2a & 4b\\ 4c & 4d \end{array}\right)$ with $a,b,c,d\in\mathbb{Z}$. This is an ideal, since it is plainly a subgroup, and $\begin{align*} \left(\begin{array}{cc} 2\alpha & 2\beta\\ 2\gamma & 2\delta\end{array}\right)\left(\begin{array}{cc} 2a & 4b\\ 4c & 4d \end{array}\right) &= \left(\begin{array}{cc} 4(\alpha a + 2\beta c) & 8(\alpha b + \beta d)\\ 4(\gamma a + 2\delta c) & 8(\gamma b + \delta d) \end{array}\right)\in J,\\ \left(\begin{array}{cc} 2a & 4b\\ 4c & 4d \end{array}\right)\left(\begin{array}{cc} 2\alpha & 2\beta\\ 2\gamma & 2\delta \end{array}\right) &= \left(\begin{array}{cc} 4(\alpha a+2\gamma b) & 4(\beta a + 2 \delta b)\\ 8(\alpha c + \gamma d) & 8(\beta c + \delta d) \end{array}\right)\in J. \end{align*}$ However, $J$ is not of the form $M_2(I)$ for an ideal $I$ of $2\mathbb{Z}$.