0
$\begingroup$

i have the following optimization problem: $$\min f(x) \\ s.t. x_i \geq 0 ,~~\sum{x_i}=1 $$

and $f(x)=\|a-B\big[ C^\top (\sum{x_i^2 D_i})C\big]^{-1} * \big[ C^\top (\sum{x_i^2 h_i})\big] \|_2^2$ where $\{B,C,D_i\}$ are matrices and $\{x,a,h_i\}$ are vectors.

I have the current value of $x_t$ and i'm interested in finding the best update step to get $x_{t+1}$ . Assume that i can calculate $\nabla_x f(x)$ and maybe also $\nabla_x^2 f(x)$, but i like to update $x_t$ also considering the constraints.

I know that the non-negativity constraint can be handled by projected-gradient update method, but not sure how to handle the other constraint.

Also i'm not sure if the Lagrangian multiplier could be useful here or not!

p.s.: Dimension of $x$ is generally smaller than 100, and even i'm more interested in sparse solutions (as minimum non-zero elements in $x$ as possible in case we can have more than one solution). About the accuracy of $x$, i think no more than 2 digits after the floating point is required.

  • 0
    How large is the $x$ vector? a few dozen entries? in the hundreds? in the hundreds of millions? How expensive is it to compute $f(x)$ and its derivatives? How accurate a solution do you need- is 3 digits good enough, or do you need an answer accurate to 15 digits? – 2017-02-27
  • 0
    @BrianBorchers: Good points! I just addressed them. However i' m not sure how expensive is to compute derivatives of $f(x)$ now, but some matrix multiplications along with matrix inverse operation would be involved in that! – 2017-02-27
  • 0
    You really can't compute values of $f(x)$? – 2017-02-28

1 Answers 1

0

This optimization is a special form of the constrained optimization: $$\min_x f(x) \\ s.t ~~g_i(x) = 0 \\ h_i(x) \leq 0$$

And one efficient way to solve it is to use SQP method which is based on the quadratic approximation of $f(x)$. Although it needs to approximate the Hessian of the Lagrangian function, as the constraints are linear the hessian is only related to $f(x)$ and can be explicitly calculated as $\nabla_{xx}^2 f(x)$.

And for a good update step for $x_t$ i can use the Powell merit function for the line-search.