An isomorphism of $RG$-modules is an $R$-linear bijection $\phi:M\to N$ which is $G$-equivariant. That is, $$\phi(g.m)=g.\phi(m).$$
If I choose a basis $\{m_1,\ldots,m_r\}$ for $M$ and $\{n_1,\ldots,n_r\}$ for $N$, I can write
$$\phi(m_j)=\sum_{i=1}^r a_{ij}n_i\;\;(1\leq j\leq n)$$
for some $a_{ij}\in R$ ($1\leq i,j\leq n$). This determines a matrix $A$.
Similarly, we can obtain matrices for left multiplication by an element $g\in G$, $\rho_M(g):M\to M$ and $\rho_N(g):N\to N$. Then, $G$-equivariance is just the statement that $A\rho_M(g)=\rho_N(g)A$.
EDIT
Okay, let's spell out the last paragraph:For each $1\leq k\leq n$,
$$g.m_k=\sum_{j=1}^nb_{jk}m_j$$
which determines a matrix $\rho_M(g)=B$, and
$$g.n_k=\sum_{j=1}^nc_{jk}n_j$$
which determines a matrix $\rho_N(g)=C$.
You want to know why equivariance implies $AB=CA$. Well
\begin{align}
\phi(g.m_k)&=\phi\left(\sum_{j=1}^kb_{jk}m_j\right)\\
&=\sum_{j=1}^nb_{jk}\phi(m_j)\\
&=\sum_{j,i=1}^nb_{jk}a_{ij}n_i\\
&=\sum_{i=1}^n\left(\sum_{j=1}^na_{ij}b_{jk}\right)n_i
\end{align}
and, similarly,
\begin{align}
g.\phi(m_k)=\sum_{i=1}^n\left(\sum_{j=1}^nc_{ij}a_{jk}\right)n_i.
\end{align}
The equality $\phi(g.m_k)=g.\phi(m_k)$ for all $k$ says that for all $i$
$$
\sum_{j=1}^na_{ij}b_{jk}=\sum_{j=1}^nc_{ij}a_{jk}.
$$
Of course, the left hand side of the equality above is the $(i,k)$-entry in $AB$ and the right hand side is the $(i,k)$-entry of $CA$. Hence $AB=CA$ as required.
