2
$\begingroup$

Suppose i have a square matrix $A_{ij}$ and a test matrix $M_{ij}$. I want to find the integral curve that joins $M_{ij}$ to $A^{-1}_{ij}$ Following the gradient descent path of the following cost function:

$$ F_{A}(M) := \sum_{ij} [( M A - I )_{ij}]^2 $$

Of course, the curve does not always end in to $A^{-1}$ because sometimes $M$ lies in a local-minima basin (i.e. not connected to the global minima).

Locally, the gradient descent path is described by the gradient of $F_{A}$

$$ \nabla_{M} F_{A} = 2 (M A - I ) A^{T} $$

or in component notation (trying very hard to not use Einstein summation notation), looks like:

$$ \frac{\partial F_{A}^{(ij)} }{ \partial M_{xy}} = 2 \sum_{k} ( \sum_{p} (M_{ip} A_{pk}) - \delta_{ik} ) A_{jk} $$

now, it is pretty clear from the above expression that its third derivative is zero.

So if i try to compute the gradient descent path to higher orders, it seems that i'm left with a quadratic curve with no further corrections.

Question: what is wrong with this approach to obtain a matrix inverse? It seems strange to me that the gradient descent curve is a simple polynomial form. Maybe i'm doing something wrong? I feel like i'm missing something very obvious, but i can't think right now what is it.

1 Answers 1