Can $(A B A^T)^{-1}$ be simplified to $A^{-T} B^{-1} A^{-1}$? If yes, why?
Edit: Would $A^T(A B A^T)^{−1}AB^{−1}$ always give me $I$?
Assuming $B$ is invertible and $A$ and $B$ are square matrices with same dimensions.
Can $(A B A^T)^{-1}$ be simplified to $A^{-T} B^{-1} A^{-1}$? If yes, why?
Edit: Would $A^T(A B A^T)^{−1}AB^{−1}$ always give me $I$?
Assuming $B$ is invertible and $A$ and $B$ are square matrices with same dimensions.
This is how inverse is defined.
If you have $(AB)^{-1}$, then that is equivalent to $B^{-1}A^{-1}$.
I suppose you need a proof?
If $A^{-1}$ and $B^{-1}$ exist, then $AA^{-1} = I$ and $BB^{-1} = I$
So, $(AB)(B^{-1}A^{-1}) = A(BB^{-1})A = AIA^{-1} = AA^{-1} = I$
and you can do it the other way to show that $(B^{-1}A^{-1}(AB)) = I$ aswell.
So you are correct.
$ (ABA^T)^{-1} = ((AB)A^T)^{-1} = A^{-T}(AB)^{-1} = A^{-T}B^{-1}A^{-1}$
In the case that $A$ and $B$ are both square matrices, yes it can, but just because $ABA^T$ is invertible (and thus square) does not imply that $A$ is even square. In the case that $A$ is not square then it cannot possibly be invertible.
$\underbrace{\begin{bmatrix}1&0\end{bmatrix}}_A\underbrace{\begin{bmatrix}1\end{bmatrix}}_B\underbrace{\begin{bmatrix}1\\0\end{bmatrix}}_{A^T}=\begin{bmatrix}1\end{bmatrix}$ is invertible however $A^{-1}$ does not exist.
In the case that all are square, one can use that $(ABA^{T})^{-1}~\text{exists}\iff \det(ABA^T)=\det(A)^2\det(B)\neq 0$
$\iff \det(A)\neq 0\wedge \det(B)\neq 0\iff A^{-1}\text{ and}~ B^{-1}~\text{exist}$
which will imply $(ABA^T)^{-1}=(A^{-1})^TB^{-1}A^{-1}$