8
$\begingroup$

how would I calculate the derivative of the following. I want to know the derivative so that I can maximise it.

$ \frac{x^TAx}{x^TBx} $

Both the matricies A and B are symmetric. I know the derivative of $\frac{d}{dx}x^TAx = 2Ax$. Haven't been very successful applying the quotient rule to the above though. Appreciate the help. Thanks!

EDIT: In response to "What goes wrong when applying the chain rule". We know that: $ \frac{d}{dx}\frac{u}{v} = \frac{vu' - uv'}{v^2} $ Which would give me: $ \frac{2x^TBxAx - 2x^TAxBx}{x^TBx^2} \, or \, \frac{2Axx^TBx - 2Bxx^TAx}{(x^TBx)^2} $

In the first case the dimensions don't agree. In the second they do, but I don't want to assume that it's correct just because the dimensions agree. If it is correct then please do let me know!

  • 1
    See edit. Thanks2012-10-13

2 Answers 2

6

Both expressions are correct, up to some missing parentheses, in the following sense: For each $i$, we have $\frac{\partial}{\partial x_i}(x^TAx) = 2(Ax)_i$ and $\frac{\partial}{\partial x_i}(x^TBx) = 2(Bx)_i$, so the quotient rule tells us that

$\frac{\partial}{\partial x_i}\frac{x^TAx}{x^TBx} = \frac{2(Ax)_i(x^TBx) - 2(Bx)_i(x^TAx)}{(x^TBx)^2}.$

This may be written, if you interpret $d/dx$ as gradient, as

$\frac{d}{dx}\frac{x^TAx}{x^TBx} = \frac{2Ax(x^TBx) - 2Bx(x^TAx)}{(x^TBx)^2} = \frac{2Axx^TBx - 2Bxx^TAx}{(x^TBx)^2},$ so your second expression is correct.

Since $(x^TAx)$ and $(x^TBx)$ are just numbers, if left inside their parentheses they can multiply a vector or matrix from either side. Hence the first expression is also correct if you are sure to leave in the parentheses:

$\frac{d}{dx}\frac{x^TAx}{x^TBx} = \frac{2(x^TBx)Ax - 2(x^TAx)Bx}{(x^TBx)^2}.$

  • 1
    Ok great, thanks. That makes sense :)2012-10-13
2

Use hypoograph/Epigraph Technique to maximize/minimize the ratio. For example

$\min_x \frac{x^T A x}{x^T B x}$

$ \equiv \min_{x,t} t$

$ \text{ subject to }$

$ {x^T A x}\leq t(x^T B x),t>0$.

Then form a Lagrangain to solve for the optimum.