No. For example, take the ring of $2\times 2$ matrices over the even integers, $2\mathbb{Z}$. Let $J$ be the subring of all matrices of the form
$$\left(\begin{array}{cc}
2a & 4b\\
4c & 4d
\end{array}\right)$$
with $a,b,c,d\in\mathbb{Z}$. This is an ideal, since it is plainly a subgroup, and
$$\begin{align*}
\left(\begin{array}{cc}
2\alpha & 2\beta\\
2\gamma & 2\delta\end{array}\right)\left(\begin{array}{cc}
2a & 4b\\
4c & 4d
\end{array}\right) &= \left(\begin{array}{cc}
4(\alpha a + 2\beta c) & 8(\alpha b + \beta d)\\
4(\gamma a + 2\delta c) & 8(\gamma b + \delta d)
\end{array}\right)\in J,\\
\left(\begin{array}{cc}
2a & 4b\\
4c & 4d
\end{array}\right)\left(\begin{array}{cc}
2\alpha & 2\beta\\
2\gamma & 2\delta
\end{array}\right) &= \left(\begin{array}{cc}
4(\alpha a+2\gamma b) & 4(\beta a + 2 \delta b)\\
8(\alpha c + \gamma d) & 8(\beta c + \delta d)
\end{array}\right)\in J.
\end{align*}$$
However, $J$ is not of the form $M_2(I)$ for an ideal $I$ of $2\mathbb{Z}$.