Approximate-Optimal Synthesis of Operational Control Systems for Dynamic Objects on the Basis of Quasilinearization and Sufficient Optimality Conditions

Abstract

A new approach to the analytical design of linear and nonlinear hierarchical control loops and multifunctional automatic control systems of real (accelerated) time, based on the combined use of dynamic-programming technologies and the quasilinearization method, is presented. For continuous dynamical systems, the fundamentals of the theory of nonlinear synthesis are presented in a formulation that allows the formation of optimal, approximately optimal, and suboptimal control strategies with respect to a vector function of optimal control that is previously unknown, but determined at small optimization lengths.

APPENDIX

The procedure for proving Theorem 3 reduces to the following. We introduce the extended state vector $y = (x,\delta \,u)$ and transform equations (2.15) and (2.16) to the form

$$\dot {y} = f(t,y) + {{{{\Gamma}}}_{1}}\vartheta ,$$

(A.1)

where $f(t,y) = (f(t,{{x}_{0}},{{u}_{0}}) + \partial \,f{\text{/}}\partial \,u \cdot \delta \,u,\,0)$ is a vector function obtained by quasilinearization according to the differential DP scheme; ${{{{\Gamma}}}_{1}} = {{\left[ {O\,\,E} \right]}^{{\text{T}}}}$ is a rectangular matrix with a “new” control vector ϑ; E₁ is the identity matrix of dimension m × m respectively; $y({{t}_{0}}) = (x({{t}_{0}}),\,0)$. Then we form the GWF for the extended space of states and controls X × U × T in the form (3.2) (${{S}_{{\text{z}}}}(y({{t}_{{\text{f}}}})) = {{S}_{{\text{z}}}}(x({{t}_{{\text{f}}}})))$.

We write out sufficient optimality conditions:

$$\mathop {\inf }\limits_{\vartheta \in {{R}^{m}}} \left( {\frac{{\partial \phi (t,y)}}{{\partial t}} + \frac{{\partial \phi (t,y)}}{{\partial y}}(f(t,y) + {{{{\Gamma}}}_{1}}\vartheta ) + {{Q}_{p}}(t,y) + 0.5\vartheta _{0}^{T}{{r}^{{ - 1}}}{{\vartheta }_{0}}} \right) = 0,$$

(A.2)

from which we determine the local-optimal “new” controls ϑ:

$$\vartheta = {{\vartheta }_{0}} = - r{{{{\Gamma}}}_{1}}\frac{{\partial \,{{\varphi }^{{\rm T}}}(t,y)}}{{\partial \,y}} = - r\frac{{\partial \,{{\varphi }^{{\rm T}}}(t,y)}}{{\partial \,\delta \,u}}.$$

(A.3)

If in the last expressions we introduce the notation ${{p}_{{\delta \,u}}} = \partial \,{{\varphi }^{{\text{T}}}}{\text{/}}\partial \,\delta u$, then condition (3.5) of the theorem is satisfied. Formula (4.3) can be obtained differently through the stationary condition: $\partial H{\text{/}}\partial \,\delta u = 0$, where H(t, y, φ_y) = $\partial \,\varphi (t,y){\text{/}}\partial \,y(f(t,y) +$ ${{{{\Gamma}}}_{1}}\vartheta ) + {{Q}_{p}}(t,y) +$ 0.5$\vartheta _{0}^{T}$r^–1ϑ₀ is the Hamiltonian of system (4.1), and $\partial \varphi (t,y){\text{/}}\partial y$ is a row vector of dimension $1 \times (n + m)$.

We formulate formula (A.3) into expression (A.2), as a result of which the sufficient optimality conditions will be written in the form of the equation

$$\frac{{\partial \,\varphi \left( {t,y} \right)}}{{\partial \,t}} + \frac{{\partial \,\varphi \left( {t,y} \right)}}{{\partial \,y}}f\left( {t,y} \right) + {{Q}_{{\text{p}}}}\left( {t,y} \right) = 0,$$

(A.4)

where the function φ(t, y) has the meaning of the Lyapunov function in stability theory.

Formula (A.4) determines the “free” motion of system (4.1). The total derivative calculated for “free” motion is calculated using the expression

$$\dot {\varphi }\left( {t,y} \right) = \frac{{\partial \,\varphi \left( {t,y} \right)}}{{\partial \,t}} + \frac{{\partial \,\varphi \left( {t,y} \right)}}{{\partial \,y}}f\left( {t,y} \right).$$

(A.5)

Equation (A.4), taking into account expression (A.5), takes the form

$$\dot {\varphi }(t,y) = - {{Q}_{{\text{p}}}}(t,y),$$

(A.6)

which implies condition (3.6) of Theorem 3.

Using the method of characteristics, we determine the solution of the partial differential equation (A.4) in the form of a canonically conjugate system [1]

$$\dot {y} = \frac{{\partial {\kern 1pt} {{H}^{{\text{T}}}}\left( {t,y,p} \right)}}{{\partial {\kern 1pt} p}} = f\left( {t,y} \right),\quad \dot {p} = - \frac{{\partial {\kern 1pt} {{H}^{{\text{T}}}}\left( {t,y,p} \right)}}{{\partial {\kern 1pt} y}} = - \frac{{\partial {\kern 1pt} {{f}^{{\text{T}}}}\left( {t,y} \right)}}{{\partial {\kern 1pt} y}}p - \frac{{\partial {\kern 1pt} Q_{{\text{p}}}^{{\text{T}}}\left( {t,y} \right)}}{{\partial {\kern 1pt} y}},$$

(A.7)

where $H\left( {t,y,p} \right) = \partial \varphi \left( {t,y} \right){\text{/}}\partial t + \partial \varphi \left( {t,y} \right){\text{/}}\partial y \cdot f\left( {t,y} \right)$ is the Hamiltonian of the “free” motion of the system (A.1), $p = \partial {{\varphi }^{{\text{T}}}}\left( {t,y} \right){\text{/}}\partial y$is a column vector of partial derivatives of the size n × m.

Revealing vectors y, p through subvectors x, δu and p_x, ${{p}_{{\delta \,u}}}$ as a result of the decomposition of relations (A.7), we obtain formulas corresponding to conditions (2.15), and (3.3)–(3.4) of Theorem 3.

Through the procedure of searching for a weak local minimum ${{u}_{{{\text{opt}}}}}(t,\tau )\mathop \to \limits_{\tau \to t} {{u}_{{{\text{opt}}}}}(t,t) = {{u}_{{\text{0}}}}(t)$ we find the optimal one in the sense of achieving a local minimum of the functional (1.4) of the process (1.3). Thus, all the conditions of Theorem 3 turn out to be satisfied. The theorem has been proven.

Proof of Theorem 4 for the differential DP scheme (Theorem 3). The equations of the canonically conjugate system (1.3), (5.1) and formula (5.4) are obtained from (2.15), (3.3) and (3.6) with u = u₀(t). The stationary conditions (5.2) and condition (3) of Theorem 2 can be deduced by following the scheme of proof from contradiction (Bliss scheme). For this, we consider the control variations determined by the gradient procedure (3.1), which, at short optimization lengths, $\Delta t$ represented by the relation $u(t)\, = {{u}_{0}}(t) + \,\vartheta \,\Delta t.$

It can be seen from the last expression that the weak local minimum (x₀, u₀) is formed through fulfillment at each stationary point of the condition u = u₀(t): exactly, by ensuring that the limit elements of minimizing sequences are equal to zero in terms of u: $\vartheta = {{\vartheta }_{0}} = 0$; and approximately, by reducing the optimization lengths: Δt → 0.

We assume the opposite, i.e., there is a control $\tilde {u}$(t) = u₀(t) + Δu(t) for which the minimum of the local criterion is less than the minimum of the local functional: $\tilde {I}$(t)< I(t). Then $d(\tilde {u}(t) - {{u}_{0}}(t)){\text{/}}dt\, = \,d\Delta u(t){\text{/}}dt,$ which contradicts the conditions of local optimality of the control in (5.10): $d\,\delta \,u{\text{/}}d\,t = 0$. In this way, $\Delta u(t) = 0$ and $\vartheta = {{\vartheta }_{0}} = 0$.

Further, it follows from formula (3.5) of Theorem 3 that, in the absence of left zero divisors, the subvector ${{p}_{{\delta \,u}}}$ of the extended skew-state vector is equal to ${{p}_{{\delta \,u}}} = 0.$

The derivative of this subvector will also be equal to zero: ${{\dot {p}}_{{\delta \,u}}} = 0,$ whence, by virtue of the relation ${{\dot {p}}_{{\delta \,u}}} = \partial \,{{H}^{{\text{T}}}}{\text{/}}\partial \,\delta \,u$ Equation (5.2) of condition 2) of Theorem 4 turns out to be valid. The theorem has been proven.

The proof of estimate (5.5) is as follows. Let $u_{0}^{j}$ be the initial approximation of the local-optimal control vector u₀, and the general recurrence relation is written from (3.1):

$$u_{0}^{{j + 1}} = u_{0}^{j} + \vartheta (u_{0}^{j})\Delta t,$$

(A.8)

where ϑ($u_{0}^{j}$) – rp_δu($u_{0}^{j}$).

To verify the validity of the estimate in (A.8) for i components of vectors $u_{0}^{{j + 1}}$, u₀, ϑ($u_{0}^{j}$), we write

$$u_{{0i}}^{{j + 1}} - {{u}_{{0i}}} = u_{{0i}}^{j} + {{\vartheta }_{i}}(u_{0}^{j})\Delta t - (u_{{0i}}^{{}} + {{\vartheta }_{i}}(u_{0}^{{}})\Delta t) = \xi _{i}^{1}(u_{0}^{j}) - \xi _{i}^{1} \left( {{{u}_{0}}} \right),$$

(A.9)

where $\xi _{i}^{1}$(u) = u_i + ϑ_i(u)Δt, ϑ_i(u₀) = –r_i$p_{{\delta u}}^{i}$(u₀) = 0, (see Theorem 4).

Expression (A.9) is represented by a Taylor series, in which we take into account the first three terms, including the residual $u_{{0i}}^{{j + 1}} - {{u}_{{0i}}} = (u_{{0i}}^{j} - {{u}_{{0i}}})\dot {\xi }_{i}^{1}\left( {{{u}_{0}}} \right) + 0.5{{(u_{{0i}}^{j} - {{u}_{{0i}}})}^{2}}\ddot {\xi }_{i}^{1}\left( \theta \right).$ Here, by the mean-value theorem (Lagrange’s theorem [4]), the remainder term is equal to

$$\int\limits_{{{u}_{{0i}}}}^{u_{{0i}}^{j}} {(u_{{0i}}^{j} - \zeta )\dot {\xi }_{i}^{1}(\zeta )d\zeta } = \frac{{{{{(u_{{0i}}^{j} - {{u}_{{0i}}})}}^{2}}}}{2}\ddot {\xi }_{i}^{1}(\theta ),$$

where θ is some value of the independent variable, intermediate between $u_{{0i}}^{{j + 1}}$, u_0i; u_0i ≤ θ ≤ $u_{{0i}}^{j}$.

Because the $\dot {\xi }_{i}^{1}$(u) = ${{\dot {u}}_{i}}$ + ${{\dot {\vartheta }}_{i}}$(u)Δt, then at the stationary points u_i = u_0i: $\dot {\xi }_{i}^{1}$(u) = 0, since u_0i is a parameter that does not vary over optimization lengths, but the condition $\dot {p}_{{\delta u}}^{i}$(u₀) = 0 follows from Theorem 4. Therefore, the estimate

$${\text{|}}u_{{0i}}^{{j + 1}} - {{u}_{{0i}}}{\text{|}} \leqslant {{k}_{{1i}}}{\text{|}}u_{{0i}}^{j} - {{u}_{{0i}}}{{{\text{|}}}^{2}},$$

(A.10)

where ${{k}_{{1i}}} = \mathop {\max }\limits_{{{u}_{0}} \leqslant \theta \leqslant u_{0}^{'}} (\ddot {\xi }_{i}^{1}\left( \theta \right){\text{/}}2)$.

Formula (A.10), written in vector form, is the first desired estimate in (5.5). The statement has been proven.