2
$\begingroup$

I am having trouble understanding the logic behind optimization of cost function of the form $$\min (|x| + |y| + |z|) \,$$ subject to constraints $$Ax \le b \qquad Cx = d $$ such as $$ x + y \le 1 \qquad 2x + z = 3.$$

I have seen methods involving representing absolute values as a new variable and putting constraints on them, $i.e$ $$|x| = a \qquad -a \le x \le a$$ but I don't understand why should we represent an equality as inequality. Aren't we changing the equation itself?

I have tried to think but am unable to grasp it. Please help.

The method is available on Wikipedia as a numerical example:- https://optimization.mccormick.northwestern.edu/index.php/Optimization_with_absolute_values

EDIT:- The relevant example

  • 0
    Eliminating $x_1$ with $x_1 = x_2 +5$ it's easy to see that the example problem is equivalent to minimizing $|x_2|$ subject to $x_2 \geq \frac{5}{3}$. Therefore the absolute value can be dropped and the solution is $x_1 = \frac{20}{3}$, $x_2 = \frac{5}{3}$ and the optimum value is $\frac{35}{3}$. – 2017-02-22
  • 0
    Do you have to solve this problem using LP? If don't, you can just solve a convex problem by changing the objective by the quadratic form $x^2+y^2+z^2$. – 2017-02-23
  • 0
    @Alex, actually yes. I just wanted to know the logic behind the method, since people have done it. – 2017-02-25

2 Answers 2

2

On that page, they are solving another problem. They are trying to make the constraint |x| < b, so it's true that they can split that inequation in those two. In your case, you can't, because you need it to be either x, or -x, not any value in between.

What you can do, is using a bivalent variable. Those are also called logical variables.

With them, you can define another variable, like u, and restrain them like this.

Let's say V is a bivalent variable. And M a big number.

x - MV ≤ u ≤ x + MV

-x - M*(1-V) ≤ u ≤ -x + M*(1-V)

This way, when V is 0, the second constraint does nothing. And the first one forces u to be x.

When V is 1, the first constraint does nothing. And the second one forces u to be -x.

Then, when the problem is solved, u can only take one of those values.

  • 0
    I still think that it represents a similar problem. I have added the example. Is this the one you were referring to? – 2017-02-22
  • 1
    On that example, U can take values that aren't right. So, it could not work as you want. The only way I know to restrain something to two values is using some bivalents variables. But maybe you're trying not to use them. If that's the case, it depends on the problem. If the function you're trying to maximize or minimize "pulls" the variable to either X or -X, then it will work. But if there's a better solution in between. Then it will not. Sometimes the problem itself won't allow that something in the middle is the best solution, so you can exploit that. But it won't work always. – 2017-02-22
  • 0
    Let's do a quick review. On that problem, if you say the best solution is x1 = 25/3 and x2 = -5/3, -25/3 < U1 <25/3 and -5/3 < U2 < 5/3. And those are both the only restraints on U. That means that optimal solution is going to be -25/3 - 3*5/3 = - 40/3. Which isn't 40/3. Maybe the values are fine anyways, maybe they are not. But it's not the same function. And we will have to do a little analysis to check if on that problem it's the same or not. – 2017-02-22
  • 0
    yes the interval should be as you mentioned, but shouldn't it be U1=25/3 and U2=5/3; which does give us 40/3 and not its negative – 2017-02-22
  • 0
    If you have any doubts of what the model does, you can always do a script on GLPK and see what it does. You can use any GLPK compiler, GUSEK on windows is pretty easy to use, and you can compile a model with a glpsol -o mySolution.sol. You'll have to add that line on GUSEK too. I'll write you a script with your model, so you can also see how it works by yourself, and try new things – 2017-02-22
  • 1
    var x1; var x2; var x3; var u1; var u2; var u3; s.t. cst0: x1+2*x2 >= 10; s.t. cst1: x1 - x2 = 5; s.t. cstu1a: -x1 <= u1; s.t. cstu1b: u1 <= x1; s.t. cstu2a: -x2 <= u2; s.t. cstu2b: u2 <= x2; s.t. cstu3a: -x3 <= u3; s.t. cstu3b: u3 <= x3; minimize z: u1 + 3*u3; end; – 2017-02-22
  • 0
    Thanks a lot for your prompt response. – 2017-02-23
4

From Boyd & Vandenberghe:

enter image description here