0
$\begingroup$

I have some difficulties in understanding some properties of Witsenhausen's counterexample in stochastic optimal control.

I thought control is about dynamical systems. However, as it seems the example here doesn't even have a single state? Isn't control always for dynamical systems?

And wouldn't it make sense to use controllers with an internal state like PID controllers? As far as I see only static mappings are used, is there a reason for this?

  • 0
    Maybe you can view it as chained discrete time state space models with input or output disturbances, with all A matrices all zeros. – 2017-01-31
  • 0
    Hm ok... what I wounder as well is how do they compute exact errors of controllers for this system? I often read that controller x has an error of y... is this just an approximation by simulating "long enough" and taking the mean error? – 2017-02-03

1 Answers 1

2

First, Witsenhaus's counterexample is intended to be the simplest example of a particular phenomenon - distributed control without communication is hard. You have a simple input, all the variables are observable. And the question is: Even in this ridiculously simple example, is linear control as good as we can do? Answer: No, nonlinear control is better even here. This is a bit of a surprise -- in very simple systems, linear control is frequently optimal. As Witsenhausen observes in the Introduction to the relevant paper,

"Considering in particular unconstrained control of linear systems with Gaussian noise and quadratic criteria, it is well known that the search for an optimum can safely be confined to the class of affine (linear plus constant) functions. This is the case for both discrete and continuous time systems"

[...]

"A counterexample is presented for which it is established that an optimal design exists and that no affine design is optimal. There does not appear to exist any counterexample involving fewer variables than the one presented here."

In the counterexample, the first input is $0 + x_0$, the zero signal with added Gaussian noise, $x_0 \sim N(0, \sigma_0^2)$. The job of the control system is to reproduce the $0$ as accurately as possible. The input noise, $x_0$ has no memory and is not a continuous process, so memory/internal state in the first controller, $C_1$, is of no use. The second controller, $C_2$, only sees a noisy copy of the output of the first controller and its job is to exactly cancel the true output of the first controller. It's noise is another Gaussian, $N(0,1)$, so memory is again useless.

So with this setup, we turn on time. The noise in the input starts flailing, the noise on the second observation starts flailing and we measure our objective function, some sort of power for the first control signal and penalty for output power. When we compute the expectation over the input noise and the second observation noise, we get an average objective power. There are nonlinear controllers that have less average power than linear controllers. Consequently, we get the surprise that even this very simple system is not optimized by linear control.

  • 0
    Thanks for the answer, however I wounder how the simulation/results are then actually obtained? Because in many papers one can read what the actual cost of a given control law is... do they just simulate the system long enough until the 10th decimal digit doesn't change any more, or how does this work with the randomness in this system? – 2017-12-09
  • 0
    @SampleTime : They do exactly what I said : They compute the expectation over the input noise and the second observation noise. Doing so leverages various techniques and some ingenuity. Generically, this is a job for calculus of variations, although insufficiently smooth control laws cause headaches. – 2017-12-09