If $N$ works then $N$ must satify that $a=a^{-1} \bmod N$ for al invertible $a$.
Proof: take an invertible $a\bmod N$. Take a prime $p$ that is congruent to $a\bmod N$ and another $q$ that is congruent to $-a^{-1}\bmod N$ (possible by dirichlet's theorem). Notice that $pq+1$ is a multiple of $N$ and therefore $\sigma(pq)=pq+1+p+q$ is also a multiple of $N$. We conclude $p\equiv-q\bmod N\implies a \equiv a^{-1}\bmod N$.
Hence the only values of $N$ that work can be $1,2,3,4,6,8,12$ and $24$. (this is a consequence of the fact that the multiplicative groups of $\mathbb Z_p^n$ is cyclic if $p$ is odd and is congruent to $\mathbb Z_{2^{n-2}}\times \mathbb Z_2$ when $p=2$).
$N=1$ clearly works.
$N=2$ does not work, although clearly it's only because of the technical case $n=1$.
$N=3$ works, because if $n\equiv -1 \bmod 3$ then one of the prime factors $p^a$ must be congruent to $-1\bmod 3$ and then we have that $\frac{p^{a+1}-1}{p-1}$ is a multiple of $3$.
$N=4$ works, because one of the prime factors $p^a$ must be congruent to $-1\bmod 4$ and this means that $p^{a+1}-1$ is a multiple of $8$ (because odd squares are $1\bmod 8$). Clearly this implies that $\frac{p^{a+1}-1}{p-1}$ is a multiple of $4$.
$N=8$ works, the proof has a couple of cases and uses the lifting the exponent lemma but it is not too hard.
Finally it is clear from the above that $6,12,24$ also all work.
So the final answer is $1,3,4,6,8,12,24$