TwoStageOptimalControl.jl

This is a Julia-package which allows for the solution of two-stage optimal control problems and vintage-structured optimal control problems.

Installation

The package is available in the Julia repository. The package can be installed by simply using the following two commands:

julia-repl julia> using Pkg julia> Pkg.add("TwoStageOptimalControl")

Documentation

For information on how to use the toolbox and some examples, please take a look into the documentation.

Problem class

This package is designed to solve problems of the form

$$ \max{C1: [0,T]\to\mathbb{R}^{n1},\quad C2:[0,T]^2\to\mathbb{R}^{n2}}\int{0}^{T} \exp(-\rho t)\Big[F1(t,C1(t),S_1(t)) + Q(t) \Big]dt + $$

$$ \hspace{4cm} + D1(S(T)) + \int0^T D2(S2(T,s))ds $$

$$ s.t. \qquad\dot{S1}(t) = g1(t,C1(t),S1(t)) ,\hspace{2cm} S1(0) = S1^0 $$

$$ \hspace{3cm} \frac{d}{dt} S2(t,s) = g2(t,s,C2(t,s),S2(t,s)) ,\qquad S2(s,s) = h(s,C1(s),S_1(s)) $$

$$ \qquad \qquad Q(t) = \int0^t F2(t,s,C2(t,s),S2(t,s)) ds $$

$$ \underline{C1}(i) \leq C{1,i}(t) \leq \overline{C1}(i) \qquad\forall i = 1,\ldots,n1 \quad,\quad\forall t\in [0,T] $$

$$ \underline{C2}(i) \leq C{2,i}(t,s) \leq \overline{C2}(i) \qquad\forall i = 1,\ldots,n2 \quad,\quad\forall s\in [0,T]\quad,\quad\forall t\in [s,T] $$

Variable and function explanations

The model consist of three different types of variables:

• concentrated state and control variables (depending on time $t$)
• distributed state and control variables (depending on time $t$ and vintage $s$)
• aggregated state variables (depending on time $t$)

Concentrated variables

The concentrated variables consist of the $n1$-dimensional control variable $C1$ and the $m1$-dimensional state variable $S1$. The initial value of $S1$ is given by $S1^0$ and its dynamics are affected by $C1$. Furthermore $C1$ also affects the initial values of the distributed variables (see below).

Distributed variables

Analogous to the concentrated variables, the distributed variables consist of the $n2$-dimensional control variable $C2$ and the $m2$-dimensional state variable $S2$. The strictly speaking partial differential equation can be solved along the characteristic lines with fixed vintage $s=\overline{s}$. While the dynamics of $S2$ are again affected by the distributed controls $C2$, the initial values of $S2$ at the beginning of each vintage $s$ are determined by the value of the concentrated state $S1$ at time $s$ and the concentrated control $C_1$.

Aggregated variables

Finally we also have an aggregated variable $Q$ which aggregates at each point in time $t$ a functional form $F_2$ over all vintages which startet before $t$.

Algorithm principles

The algorithm uses a gradient based approach following a sequence of steps:

1. Start with a given guess for the optimal solution of the control variables.
2. Calculate the corresponding profiles of the state variables and co-state variables (based on the maximum principle)
3. Calculate the gradient based on the Hamiltonian.
4. Adjust the guess for the control in the direction of the gradient.
5. Find the optimal adjustment step in the direction of the gradient.
6. Define new currently best solution.
7. Iterate from step 2 until no improvement can be found anymore.

Two-stage optimal control problems with random switching time

This package is based on a project about the solution of two-stage optimal control problems with random switching time. This problem class takes the following form (with $\mathbb{E}$ being the expected value with respect to $\tau$):

$$ \max{C1: [0,T]\to\mathbb{R}^{n1}}\mathbb{E}\int{0}^{\tau} \exp(-\rho t) F1(t,C1(t),S1(t)) dt + \exp(-\rho\tau)V^*(\tau,C1(\tau),S_1(\tau)) $$

$$ s.t. \qquad\dot{S1}(t) = g1(t,C1(t),S1(t)) ,\hspace{2cm} S1(0) = S1^0 $$

$$ \underline{C1}(i) \leq C{1,i}(t) \leq \overline{C1}(i) \qquad\forall i = 1,\ldots,n1 $$

with $V^*(\tau,S_1(\tau))$ being the value-function of the second stage problem

$$ V^*(\tau,S1(\tau)) = \max{C2:[\tau,T]\to\mathbb{R}^{n2}}\int{\tau}^T \exp(-\rho(t-\tau))F2(t,\tau,C2(t),S2(t)) dt $$

$$ \hspace{3cm} \dot{S2}(t) = g2(t,\tau,C2(t),S2(t)) ,\qquad S2(\tau) = h(\tau,C1(\tau),S_1(\tau)) $$

$$ \underline{C2}(i) \leq C{2,i}(t) \leq \overline{C2}(i) \qquad\forall i = 1,\ldots,n2 \quad,\quad\forall t\in [\tau,T] $$

and $\tau$ being the random switching time between the two stages with the stochastic properties being best described by the $Z(t):= \mathbb{P}\left[\tau > t\right]$, the probability of surviving in the first stage. The hazard rate $\eta$ of $Z$ thereby can depend on time $t$ the state variables $S1$ and the control variablee $C1$.

$$ \dot{Z}(t) = -\eta(t,C1,S1)\cdot Z(t) $$

Overall this problem type can be transformed into a vintage-structed optimal control problem presented above. For details see the work of Wrzaczek, Kuhn and Frankovic (2020).

Planned extensions of the toolbox

1. Generalise to the age-structured optimal control problems with initial distribution of state variables, intial and boundary controls, and more generalised version of aggregated variables.

2. Allow for more general/complicated feasible regions for the control variables.

3. Include the option to define equilibrium conditions, which the solution has to fulfill.