Conditions for optimal control

Question

Consider the following system: $\text{minimize}_{u} \text{ } J = \int_0^\infty u^2 dt$

Subject to: $\dot{x} = Ax + Bu, \text{ } x(0) = x_0$

Where: $A = \begin{bmatrix} k & 1 \\ 0 & 1 \end{bmatrix}$, $B = \begin{bmatrix} 0 \\ 1 \end{bmatrix}$.

What are the conditions for $k$ such that the optimal control exists?

I can show that the Riccati-equation: $A^TP + PA - PBB^TP = 0$ has only symmetric positive-definite solutions $P$ when $k>0$.

Can an optimal control exist if $P$ is not a symmetric positive-definite solution?

Any help appreciated!

Answer 1

Consider there is a $P$ such that $$A^T P+PA-PBB^TP=0$$ then the state feedback is $K=B^TP$ and the closed-loop state matrix would be: $$A-BK=A-BB^TP$$ Needless to say, an optimal control has to stabilize the system. Which means the closed-loop state matrix must be Hurwitz. Now the non-trivial solutions to the Riccati equation (regardless of being positive-definite) are: $$\frac 12P=\pmatrix{0&0 \\ 0&1},\; \pmatrix{k(k-1)^2& k(k-1)\\ k(k-1)&k},\; \pmatrix{k(k+1)^2&k(k+1)\\ k(k+1)&k+1}$$ Hence the closed-loop state matrix would be: $$A_c=\pmatrix{k&1 \\ 0&-1},\quad \pmatrix{k& 1\\ 2k(1-k)&1-2k},\quad\pmatrix{k&1\\ -2k(k+1)&-2k-1}$$ You can see that the eigenvalues of the first matrix are $\{-1,k\}$, the second one $\{1,-k\}$ and for the third one $\{-1,-k\}$. So the second one is unstable and the stability conditions are $k<0$ for the first $P$ and $k>0$ for the third $P$.

In conclusion, the first $P$ is positive semidefinite and the third $P$ is positive definite for $k>0$. So the optimal policy depends on the poles of the open-loop system.