0
$\begingroup$

I have the following optimal control problem, which can be considered in a two period discrete time setting. It has this generic form.

Let $w: \mathbb{R} \rightarrow \mathbb{R}$ be a given function, modeling terminal reward (in my example in mind, $w$ is negative, bounded, increasing, and concave). At time $0$, the state of the system is known, and equal to $x$. The controller can choose a control $t \in [0,\infty )$ such that the state of the system at time $1$ is $f(t,x,Z)$, where $Z$ is a standard random variable. The cost of using strategy $t$ is $g(t,x)$. The goal is to calculate

$\hat{w}(x) = \underset{t}{min} \ \left( g(t,x) + e^{-\lambda t} E [w(f(t,x,Z))] \right),$

and find the optimal $t$. Is there a reference for explicit/computable solutions to this problem? Subject to certain restrictions on w,f,g?

  • 0
    I would consider this to be a problem of minimizing an expected value (ie, $\min_t E[g(t,x) + e^{-\lambda t} w(f(t,x,Z))]$) rather than an optimal control problem. The solution techniques depend on how you evaluate the expectation.2012-09-25
  • 0
    can you elaborate a little bit on your last sentence?2012-09-25
  • 0
    Are you using Monte Carlo or do you have a way of explicitly evaluating the expectation? If the former, you might look up work by Georg Pflug, if the latter, then this is an 'ordinary' optimization problem.2012-09-25
  • 0
    The expectation can't be computed analytically. How can this be solved using monte carlo? Is there another numeric method for solving it without probability?2012-09-25
  • 0
    I'm not sure how you can avoid probability if you have an expectation that cannot be computed explicitly?2012-09-25
  • 0
    Thanks copper.hat. Do you have a reference for setting up Monte Carlo for this?2012-09-25
  • 0
    Sorry, I don't have one I can recommend. I mentioned an author above (Pflug) who has a recent (relatively) book on stochastic optimization, but I have only glanced at his work. The bibliography might provide some clues.2012-09-25

0 Answers 0