The turnpike property and the longtime behavior of the Hamilton–Jacobi–Bellman equation for finite-dimensional LQ control problems

Abstract

We analyze the consequences that the so-called turnpike property has on the longtime behavior of the value function corresponding to a finite-dimensional linear-quadratic optimal control problem with general terminal cost and constrained controls. We prove that, when the time horizon T tends to infinity, the value function asymptotically behaves as $W(x) + c\, T + \lambda$, and we provide a control interpretation of each of these three terms, making clear the link with the turnpike property. As a by-product, we obtain the longtime behavior of the solution to the associated Hamilton–Jacobi–Bellman equation in a case where the Hamiltonian is not coercive in the momentum variable. As a result of independent interest, we showed that linear-quadratic optimal control problems with constrained control enjoy a turnpike property, also particularly when the steady optimum may saturate the control constraints.

Appendices

Appendix A. Proof of the turnpike property

This appendix is devoted to the proof of the turnpike property stated in Theorem 1.2. The main difficulty resides in the fact that we are considering the constrained control case. In addition, we do not make the assumption of the steady optimal control $\overline{u}$ being at the interior of the control set U, which would make the proof much easier, and moreover would allow us to prove turnpike with an exponential rate.

We start by proving the following crucial Lemma which is a direct consequence of [40,  Remark 2.1] and we include its proof for self-consistency.

Lemma A.1

Assume (A, C) is detectable and take $f\in L^2(0,T;{\mathbb {R}}^n)$. Then, there exists a constant $K=K\left( A,C\right) \ge 0$, independent of T and f, such that for any $T\ge 1$ and for any y solution to

$$\begin{aligned} \frac{\hbox {d}}{\hbox {d}s}y=Ay+f\quad \text{ in } \quad (0,T), \end{aligned}$$

(A.1)

we have

$$\begin{aligned} \Vert y(t)\Vert ^2+\int _0^T\Vert y\Vert ^2\hbox {d}s\le K \left[ \Vert y(0)\Vert ^2+\int _0^T\Vert C\, y\Vert ^2\hbox {d}s +\int _0^T\Vert f\Vert ^2 \hbox {d}s\right] , \end{aligned}$$

(A.2)

for any $t\in [0,T]$.

Proof

In the present proof, K will denote a (sufficiently large) constant depending only on (A, C).

Step 1 Decomposition into stable and antistable part

Following the notation of [15], ${\mathscr {L}}^{-}(A)$ and ${\mathscr {L}}^{0+}(A)$ denote resp. the A-invariant subspaces of ${\mathbb {R}}^n$ spanned by the generalized eigenvectors of A corresponding to eigenvalues $\lambda$ of A such that $\text{ Re }(\lambda )<0$ and $\text{ Re }(\lambda )\ge 0$. By linear algebra,

$$\begin{aligned} {\mathbb {R}}^n={\mathscr {L}}^{-}(A)\oplus {\mathscr {L}}^{0+}(A), \end{aligned}$$

where $\oplus$ stands for the direct sum. Then, let y be a solution to (A.1). Denote by $y_1$ and $y_2$ resp. the projections of y onto ${\mathscr {L}}^{-}(A)$ and ${\mathscr {L}}^{0+}(A)$. Then, $y=y_1+y_2$ and, for $i=1,2$,

$$\begin{aligned} \frac{\hbox {d}}{\hbox {d}s}y_i=Ay_i+f_i\quad \text{ in } \quad (0,T), \end{aligned}$$

where $f_1$ and $f_2$ stand for resp. the projection of f onto ${\mathscr {L}}^{-}(A)$ and ${\mathscr {L}}^{0+}(A)$.

Step 2 Estimate for the asymptotically stable part

We have

$$\begin{aligned} \frac{\hbox {d}}{\hbox {d}s}y_1=Ay_1+f_1\quad \text{ in } \quad (0,T), \end{aligned}$$

All the eigenvalues of $L_A \restriction _{{\mathscr {L}}^{-}(A)}$ are strictly negative, where we have denoted by $L_A$ the linear operator associated with the matrix A. Then, we have, for any $s\in [0,T]$

$$\begin{aligned} \Vert y_1(s)\Vert +\int _0^T\Vert y_1\Vert ^2\hbox {d}s\le & {} K \left[ \Vert y_1(0)\Vert +\Vert f_1\Vert _{L^2(0,T;{\mathbb {R}}^n)}\right] \nonumber \\\le & {} K\left[ \Vert y(0)\Vert +\Vert f\Vert _{L^2(0,T;{\mathbb {R}}^n)}\right] , \end{aligned}$$

(A.3)

the constant K depending only on A.

Step 3 An observability inequality in the time interval [0, 1]

To proceed with the antistable part, we shall first prove the existence of an observability constant $K=K(A,C)\ge 0$, such that for any $\tilde{y}\in H^1(0,T;{\mathscr {L}}^{0+}(A))$ solution to

$$\begin{aligned} \frac{\hbox {d}}{\hbox {d}s}\tilde{y}=A\tilde{y}\quad \text{ in } \quad (0,1), \end{aligned}$$

we have

$$\begin{aligned} \left\| \tilde{y}\right\| _{L^{\infty }(0,1)}+\left\| \tilde{y} \right\| _{L^{2}(0,1)}\le K\left\| C\tilde{y}\right\| _{L^{2}(0,1)}. \end{aligned}$$

(A.4)

To that end, define

$$\begin{aligned} \left\| \cdot \right\| _{a}:{\mathscr {L}}^{0+}(A) \longrightarrow {\mathbb {R}}^+,\quad \left\| x\right\| _{a}:=\left\| \tilde{y}_x \right\| _{L^{\infty }(0,1)}+\left\| \tilde{y}_x\right\| _{L^{2}(0,1)} \end{aligned}$$

and

$$\begin{aligned} \left\| \cdot \right\| _{b}:{\mathscr {L}}^{0+}(A) \longrightarrow {\mathbb {R}}^+,\quad \left\| x\right\| _{b}:=\left\| C\tilde{y}_x\right\| _{L^{2}(0,1)}, \end{aligned}$$

where $\tilde{y}_x$ solves

$$\begin{aligned} {\left\{ \begin{array}{ll} \frac{\hbox {d}}{\hbox {d}s}\tilde{y}_x(s) = A\, \tilde{y}_x(s), &{} s\in (0,1) \\ \tilde{y}_x (0) = x. \end{array}\right. } \end{aligned}$$

Now, using that, being (A, C) is detectable, all the modes in ${\mathscr {L}}^{0+}(A)$ are observable (see definition of detectability in [15, at the bottom of page 232]), we deduce that both $\left\| \cdot \right\| _{a}$ and $\left\| \cdot \right\| _{b}$ are norms on the subspace ${\mathscr {L}}^{0+}(A)$. Since ${\mathscr {L}}^{0+}(A)$ is finite-dimensional, they are equivalent, whence (A.4) follows.

Step 4 Estimate for the antistable part

By definition

$$\begin{aligned} \frac{\hbox {d}}{\hbox {d}s}y_2=Ay_2+f_2\quad \text{ in } \quad (0,T), \end{aligned}$$

Consider an arbitrary interval $[a,b]\subset [0,T]$, with length $\left| b-a\right| =1$. By step 3, we have

$$\begin{aligned} \left\| y_2\right\| _{L^{\infty }(a,b)}^2 +\left\| y_2\right\| _{L^{2}(a,b)}^2\le K \left[ \int _{a}^b\Vert C\, y_2\Vert ^2 \hbox {d}s+\int _{a}^{b}\Vert f_2\Vert ^2 \hbox {d}s\right] . \end{aligned}$$

On the one hand, due to the arbitrariness of [a, b], this yields

$$\begin{aligned} \left\| y_2\right\| _{L^{\infty }(0,T)}^2\le K \left[ \int _{0}^T\Vert C\, y_2\Vert ^2 \hbox {d}s+\int _{0}^T\Vert f_2\Vert ^2 \hbox {d}s\right] . \end{aligned}$$

On the other hand,

$$\begin{aligned} \int _0^T\left\| y_2\right\| ^2\hbox {d}s\le & {} \sum _{i=0}^{\lfloor {T}\rfloor } \int _i^{i+1}\left\| y_2\right\| ^2\hbox {d}s+\int _{T-1}^T \left\| y_2\right\| ^2\hbox {d}s\nonumber \\\le & {} K\sum _{i=0}^{\lfloor {T}\rfloor } \left[ \int _{i}^{i+1}\Vert C\, y_2\Vert ^2 \hbox {d}s+\int _{i}^{i+1} \Vert f_2\Vert ^2 \hbox {d}s\right] \nonumber \\&+K\left[ \int _{T-1}^{T}\Vert C\, y_2\Vert ^2 \hbox {d}s +\int _{T-1}^{T}\Vert f_2\Vert ^2 \hbox {d}s\right] \nonumber \\\le & {} K\left[ \int _{0}^T\Vert C\, y_2\Vert ^2 \hbox {d}s+\int _{0}^T\Vert f_2\Vert ^2 \hbox {d}s\right] . \end{aligned}$$

Then, for any $t\in [0,T]$, we have

$$\begin{aligned} \Vert y_2(t)\Vert ^2+\int _0^T\left\| y_2\right\| ^2\hbox {d}s\le & {} K\left[ \int _{0}^T\Vert C\, y_2\Vert ^2 \hbox {d}s +\int _{0}^T\Vert f_2\Vert ^2 \hbox {d}s\right] \nonumber \\\le & {} K\left[ \int _{0}^T\Vert C\, y_2\Vert ^2 \hbox {d}s +\int _{0}^T\Vert f\Vert ^2 \hbox {d}s\right] \nonumber \\\le & {} K\left[ \int _{0}^T\Vert C\, y\Vert ^2 \hbox {d}s +\int _{0}^T\Vert C\, y_1\Vert ^2 \hbox {d}s+\int _{0}^T\Vert f\Vert ^2 \hbox {d}s\right] \nonumber \\\le & {} K\left[ \Vert y(0)\Vert ^2+\int _{0}^T\Vert C\, y\Vert ^2 \hbox {d}s +\int _{0}^T\Vert f\Vert ^2 \hbox {d}s\right] , \end{aligned}$$

(A.5)

where in the last inequality we have employed (A.3).

Step 4 Conclusion

Putting together (A.3) and (A.5), we conclude. $\square$

Remark A.2

Observe that, assuming that(A, C) is detectable, we have, for some $\beta = \beta (A,C)>0$, the inequality

$$\begin{aligned} \Vert y_s\Vert ^2\le \beta \left[ \Vert Ay_s\Vert ^2+\Vert C\, y_s\Vert ^2\right] , \qquad \forall y_s\in {\mathbb {R}}^n, \end{aligned}$$

(A.6)

This is a consequence of inequality (A.2) applied to the trajectory $\tilde{y}(t):=ty_s$, (see [40]). The steady inequality (A.6) yields strict convexity of $J_s$ and hence uniqueness of the minimizer for the stationary optimal control problem (1.4).

Remark A.3

In Definition 1.1, by using $u-u_1\in L^1(0,+\infty ;{\mathbb {R}}^m)$ and $y-y_1\in L^1(0,+\infty ;{\mathbb {R}}^n)$ together with $\frac{d}{ds}\left( y\left( s\right) -y_1\left( s\right) \right) =A\left( y\left( s\right) -y_1\left( s\right) \right) +B\left( u\left( s\right) -u_1\left( s\right) \right)$, $s\in (0,+\infty )$,

the solution y stabilizes toward $y_1$, i.e., $y\left( t\right) -y_1\left( t\right) \underset{t\rightarrow +\infty }{\longrightarrow }0$.

Note also that U-stabilizability follows from exact controllability under the control constraint $u\left( t\right) \in U$ (see, e.g., [13, 35, 44]). In case $U={\mathbb {R}}^m$, the U-stabilizability is equivalent to (unconstrained) exponential stabilizability of (A, B) [10,  Remark 2.2 page 24].

We now prove the following result, which provides an upper bound $\left\| y_{_{T}}\right\|$ uniform in T, and also gives the inequality (1.9) from Theorem 1.2.

Lemma A.4

There exists $K=K(A,B,C,U,x,z,g)$ such that, for any $T\ge 1$ and for every $t\in [0,T]$, we have

$$\begin{aligned} \left\| y_{_{T}}\left( t\right) \right\| \le K \end{aligned}$$

(A.7)

and

$$\begin{aligned} \int _0^T \left[ \Vert u_{_{T}}(s)-\overline{u}\Vert ^2 +\Vert y_{_{T}}(s)-\overline{y}\Vert ^2\right] \hbox {d}s\le K \end{aligned}$$

(A.8)

Proof of Lemma A.4

By Lemma A.1 applied to $y-\overline{y}$, we have

$$\begin{aligned} \Vert y_{_{T}}(t)-\overline{y}\Vert ^2\le K\left[ \Vert x-\overline{y}\Vert ^2 +\int _0^T \left[ \Vert u_{_{T}}(s)-\overline{u}\Vert ^2 + \Vert C \, \left( y_{_{T}}(s)-\overline{y}\right) \Vert ^2\right] \hbox {d}s\right] , \end{aligned}$$

whence

$$\begin{aligned} \dfrac{1}{2} \int _0^T \left[ \Vert u_{_{T}}(s) -\overline{u}\Vert ^2 + \Vert C \, \left( y_{_{T}}(s) -\overline{y}\right) \Vert ^2\right] \hbox {d}s\ge \alpha \Vert y_{_{T}}(s)-\overline{y}\Vert ^2-K, \end{aligned}$$

where $\alpha = \alpha (A,C)>0$ and $K=K(A,B,C,x,z)\ge 0$. Using the above inequality, together with Lemma 2.2 (inequality (2.3)), yields

$$\begin{aligned} J_{T,x}(u_{_{T}})-TV_s= & {} \int _{0}^{T} \left[ \dfrac{1}{2}\Vert u(s)\Vert ^{2} + \dfrac{1}{2} \Vert C\, y_{_{T}}(s) - z\Vert ^{2} - V_s \right] \;\hbox {d}s\nonumber \\\ge & {} \left[ \dfrac{1}{2} \int _0^T \left[ \Vert u_{_{T}}(s) -\overline{u}\Vert ^2 + \Vert C \, \left( y_{_{T}}(s) -\overline{y}\right) \Vert ^2\right] \hbox {d}s\right. \nonumber \\&\quad \quad \quad +(\overline{p},x-y_{_{T}} (T))_{{\mathbb {R}}^n}+ g(y_{_{T}}(T))\bigg ]\nonumber \\\ge & {} \left[ \dfrac{1}{2} \int _0^T \left[ \Vert u_{_{T}}(s) -\overline{u}\Vert ^2 + \Vert C \, \left( y_{_{T}}(s) -\overline{y}\right) \Vert ^2\right] \hbox {d}s\right. \nonumber \\&-K\left( 1+\left\| y_{_{T}}(T) -\overline{y}\right\| \right) \bigg ]\nonumber \\\ge & {} \left[ \alpha \Vert y_{_{T}}(T)-\overline{y}\Vert ^2 -K\left( \Vert y_{_{T}}(T)-\overline{y}\Vert +2\right) \right] \nonumber \\\ge & {} \dfrac{\alpha }{2} \Vert y_{_{T}}(T)-\overline{y}\Vert ^2-K. \end{aligned}$$

(A.9)

Now, by U-stabilizability, there exists a control $\hat{u}\in L^2(0,+\infty ;U)$, such that

$$\begin{aligned}&\hat{u}-\overline{u}\in L^2(0,+\infty ; {\mathbb {R}}^m)\cap L^1(0,+\infty , {\mathbb {R}}^m),\\&\hat{y}-\overline{y}\in L^2(0,+\infty ;{\mathbb {R}}^n) \cap L^1(0,+\infty ; {\mathbb {R}}^n), \end{aligned}$$

where $\hat{y}$ is the solution to (1.1), with initial datum x and control $\hat{u}$. Therefore

$$\begin{aligned} J_{T,x}(u_{_{T}})-TV_s\le & {} J_{T,x}(\hat{u})-TV_s\nonumber \\= & {} \dfrac{1}{2} \int _0^T \left[ \Vert \hat{u}(s)-\overline{u}\Vert ^2 + \Vert C \, \left( \hat{y} (s)-\overline{y}\right) \Vert ^2\right] \hbox {d}s\nonumber \\&+\int _0^T \left[ \left( \overline{u},\hat{u}(s) -\overline{u}\right) _{{\mathbb {R}}^m} +\left( C\overline{y}-z,\, C \left( \hat{y}(s) -\overline{y}\right) \right) _{{\mathbb {R}}^n}\right] \hbox {d}s + g(\hat{y}(T))\nonumber \\\le & {} K, \end{aligned}$$

(A.10)

where $=K(A,B,C,U,x,z,g)$. Hence, putting together (A.9) and (A.10), we get

$$\begin{aligned} \dfrac{\alpha }{2} \Vert y_{_{T}}(T)-\overline{y}\Vert ^2-K\le & {} J_{T,x}(u_{_{T}})-TV_s\\\le & {} J_{T,x}(\hat{u})-TV_s\le K, \end{aligned}$$

i.e., the desired boundedness for $y_{_{T}}$.

Moreover, by inequality (2.3), the boundedness from below of g together with (A.10), we have

$$\begin{aligned} \int _0^T \left[ \Vert u_{_{T}}(s)-\overline{u}\Vert ^2 +\Vert C \, \left( y_{_{T}}(s)-\overline{y}\right) \Vert ^2\right] \hbox {d}s&\le J_{T,x}(u_{_{T}})-TV_s\nonumber \\&\quad +(\overline{p},-x+y_{_{T}}(T))_{{\mathbb {R}}^n} -g\left( y_{_{T}}\left( T\right) \right) \\&\le K. \end{aligned}$$

Then, inequality (A.8) follows from an application of Lemma A.1. This finishes the proof. $\square$

We now prove the validity of the turnpike property.

Proof of Theorem 1.2

Inequality (1.9) has already been proved in Lemma A.4. It remains to prove (1.8). Throughout this proof, K will always denote a (sufficiently large) constant depending only on A, B, C, U, x, z and g.

By (1.9), for any $\eta \in (0,1)$, there exists $\zeta =\zeta (A,B,C,U,x,z,g,\eta )>0$ such that, for all $T>\zeta$, we have

$$\begin{aligned} \frac{1}{\zeta }\int _0^{\zeta } \left[ \Vert u_{_{T}}(s) {-}\overline{u}\Vert ^2 {+} \Vert y_{_{T}}(s)-\overline{y}\Vert ^2\right] \hbox {d}s\le & {} \frac{1}{\zeta }\int _0^T \left[ \Vert u_{_{T}}(s) {-}\overline{u}\Vert ^2 {+} \Vert y_{_{T}}(s)-\overline{y}\Vert ^2\right] \hbox {d}s\\\le & {} \frac{K}{\zeta }<\eta ^2 \end{aligned}$$

and

$$\begin{aligned} \frac{1}{\zeta }\int _{T-\zeta }^{T} \left[ \Vert u_{_{T}}(s) {-}\overline{u}\Vert ^2 {+} \Vert y_{_{T}}(s)-\overline{y}\Vert ^2\right] \hbox {d}s\le & {} \frac{1}{\zeta }\int _{0}^T \left[ \Vert u_{_{T}}(s) {-}\overline{u}\Vert ^2 {+} \Vert y_{_{T}}(s)-\overline{y}\Vert ^2\right] \hbox {d}s\\\le & {} \frac{K}{\zeta }<\eta ^2. \end{aligned}$$

By integral mean value theorem, for any $T\ge 1+2\zeta$, there exist $t_{T,1}\in [0,\zeta ]$ and $t_{T,2}\in \left[ T-\zeta ,T\right]$, such that

$$\begin{aligned} \left\| u_{_{T}}\left( t_{T,1}\right) {-}\overline{u}\right\| ^2 {+}\left\| y_{_{T}}\left( t_{T,1}\right) -\overline{y}\right\| ^2 {=}\frac{1}{\zeta }\int _0^{\zeta } \left[ \Vert u_{_{T}}(s){-}\overline{u}\Vert ^2 {+} \Vert y_{_{T}}(s){-}\overline{y}\Vert ^2\right] \hbox {d}s<\eta ^2 \end{aligned}$$

(A.11)

and

$$\begin{aligned} \left\| u_{_{T}}\left( t_{T,2}\right) {-}\overline{u}\right\| ^2 {+}\left\| y_{_{T}}\left( t_{T,2}\right) {-}\overline{y}\right\| ^2 {=}\frac{1}{\zeta }\int _{T-\zeta }^{T} \left[ \Vert u_{_{T}}(s) {-}\overline{u}\Vert ^2 {+} \Vert y_{_{T}}(s){-}\overline{y}\Vert ^2\right] \hbox {d}s{<}\eta ^2. \end{aligned}$$

(A.12)

By the U-stabilizability, there exists a control $\tilde{u}\in L^2(t_{T,1},+\infty ;U)$, such that

$$\begin{aligned} \tilde{u}-\overline{u}\in L^1(t_{T,1},+\infty ; {\mathbb {R}}^m) \cap L^2(t_{T,1},+\infty ; {\mathbb {R}}^m) \end{aligned}$$

and its associated trajectory $\tilde{y}$, solution to (1.1) with initial condition $y_{_T}(t_{T,1})$ satisfies

$$\begin{aligned} \tilde{y}-\overline{y}\in L^1(t_{T,1},+\infty ;{\mathbb {R}}^n) \cap L^2(t_{T,1},+\infty ;{\mathbb {R}}^n), \end{aligned}$$

with estimates

$$\begin{aligned}&\left\| \tilde{u}-\overline{u}\right\| _{L^1(t_{T,1},+\infty ) \cap L^2(t_{T,1},+\infty )} \le \gamma \left\| y_{_{T}} \left( t_{T,1}\right) -\overline{y}\right\| ,\nonumber \\&\left\| \tilde{y}-\overline{y}\right\| _{L^1(t_{T,1},+\infty ) \cap L^2(t_{T,1},+\infty )}\le \gamma \left\| y_{_{T}} \left( t_{T,1}\right) -\overline{y}\right\| \end{aligned}$$

(A.13)

the constant $\gamma$ depending only on $\left( A,B,U\right)$. Set

$$\begin{aligned} {\left\{ \begin{array}{ll} \dot{\hat{y}}(s) = A\, \hat{y}(s) + B\, \hat{u}(s), &{} s\in \left( 0,T\right) \\ \hat{y}\left( 0\right) = x, \end{array}\right. } \qquad \hat{u}\left( s\right) :={\left\{ \begin{array}{ll} u_{_{T}}\left( s\right) \quad &{} s\in \left( 0,t_{T,1}\right) \\ \tilde{u}\left( s\right) \quad &{} s \in \left( t_{T,1},t_{T,2}\right) \\ u_{_{T}}(s) \quad &{} s\in \left( t_{T,2},T\right) . \end{array}\right. } \end{aligned}$$

(A.14)

By using (A.13) and (A.14), we get

$$\begin{aligned} \left\| \hat{y}\left( t_{T,2}\right) -\overline{y}\right\| \le \gamma \left\| y_{_{T}}\left( t_{T,1}\right) -\overline{y}\right\| , \end{aligned}$$

(A.15)

with $\gamma =\gamma \left( A,B,U\right)$.

Now, let us define the functional

$$\begin{aligned} Q(u) :=\dfrac{1}{2} \int _{t_{T,1}}^{t_{T,2}} \left[ \Vert u(s)\Vert ^2 + \Vert C \, y (s)-z\Vert ^2\right] \hbox {d}s, \end{aligned}$$

(A.16)

defined for any $u\in L^2\left( t_{T,1},t_{T,2};U\right)$, where $y(\cdot )$ is the solution to (1.1) with control u and initial condition $y(t_{T,1}) = y_{_T} (t_{T,1})$.

Let us estimate from above the following quantity

$$\begin{aligned} \Lambda := \frac{1}{2}\int _{t_{T,1}}^{t_{T,2}} \left[ \Vert u_{_{T}}(s)-\overline{u}\Vert ^2 + \Vert C \, \left( y_{_{T}}(s)-\overline{y}\right) \Vert ^2\right] \hbox {d}s. \end{aligned}$$

By adapting the techniques of Lemma 2.2 to the functional Q in (A.16), we obtain the analogous version of (2.3), which reads as

$$\begin{aligned} \Lambda \le Q(u_{_{T}})-\left( t_{T,2}-t_{T,1}\right) V_s -\left( \overline{p},y_{_{T}}\left( t_{T,1}\right) -y_{_{T}} \left( t_{T,2}\right) \right) _{{\mathbb {R}}^n}. \end{aligned}$$

Now, using the definition of $J_{T,x}$ and Q, along with the fact that $u_{_T}$ minimizes $J_{T,x}$, we deduce

$$\begin{aligned} \Lambda&\le J_{T,x}(u_{_{T}})-\int _{[0,t_{T,1}] \cup [t_{T,2},T]} \left[ \Vert u_{_{T}}(s)\Vert ^2 + \Vert C \, y_{_{T}} (s)-z\Vert ^2\right] \hbox {d}s-g\left( y_{_{T}}(T)\right) \\&\quad -\left( t_{T,2}-t_{T,1}\right) V_s-\left( \overline{p}, y_{_{T}}\left( t_{T,1}\right) -y_{_{T}}\left( t_{T,2}\right) \right) _{{\mathbb {R}}^n} \\&\le J_{T,x}(\hat{u})-\int _{[0,t_{T,1}]\cup [t_{T,2},T]} \left[ \Vert u_{_{T}}(s)\Vert ^2 + \Vert C \, y_{_{T}} (s)-z\Vert ^2\right] \hbox {d}s-g\left( y_{_{T}}(T)\right) \\&\quad -\left( t_{T,2}-t_{T,1}\right) V_s-\left( \overline{p}, y_{_{T}}\left( t_{T,1}\right) -y_{_{T}}\left( t_{T,2}\right) \right) _{{\mathbb {R}}^n}. \end{aligned}$$

Using the definition of Q in (A.16) and the choice of the control $\hat{u}$ in (A.14), we get

$$\begin{aligned} \Lambda&\le Q(\hat{u})+\int _{t_{T,2}}^T \left[ \Vert C \, \hat{y} (s)-z\Vert ^2 - \Vert C \, y_{_{T}} (s)-z\Vert ^2\right] \hbox {d}s \nonumber \\&\quad +g\left( \hat{y}(T)\right) -g\left( y_{_{T}}(T)\right) -\left( t_{T,2}-t_{T,1}\right) V_s-\left( \overline{p},y_{_{T}}\left( t_{T,1}\right) -y_{_{T}}\left( t_{T,2}\right) \right) _{{\mathbb {R}}^n}. \end{aligned}$$

(A.17)

Now, noting that $u_{_T}$ and $\hat{u}$ coincide in the interval $(t_{T,2} , T)$, and that $T-t_{T,2} \le \zeta$, we can use Gronwall’s inequality to estimate

$$\begin{aligned} \Vert \hat{y}(T) - y_{_T}(T)\Vert \le C \Vert \hat{y} (t_{T,2}) -y_{_T} (t_{T,2}) \Vert , \end{aligned}$$

where C is independent of T. Moreover, using the local Lipschitz continuity of g and (A.7), we obtain

$$\begin{aligned} |g\left( \hat{y}(T)\right) -g\left( y_{_{T}}(T)\right) | \le K \Vert \hat{y} (t_{T,2}) - y_{_T} (t_{T,2}) \Vert . \end{aligned}$$

(A.18)

Then, from the estimate (A.17), an analogous version of the identity (2.2) for the functional Q, combined with (A.17), (A.13) and (A.15), yields

$$\begin{aligned} \Lambda&\le \int _{t_{T,1}}^{t_{T,2}} \left[ \Vert \hat{u}(s)-\overline{u}\Vert ^2 + \Vert C \, \left( \hat{y}(s)-\overline{y}\right) \Vert ^2\right] \hbox {d}s \\&\quad +\int _{t_{T,1}}^{t_{T,2}} \left[ \left( \overline{u}, \hat{u}(s)-\overline{u}\right) _{{\mathbb {R}}^m} +\left( C\overline{y}-z,\, C \left( \hat{y} (s) -\overline{y}\right) \right) _{{\mathbb {R}}^n}\right] \hbox {d}s\\&\quad -\left( \overline{p},y_{_{T}}\left( t_{T,1}\right) -y_{_{T}}\left( t_{T,2}\right) \right) _{{\mathbb {R}}^n} +K\left[ \left\| y_{_{T}}\left( t_{T,1}\right) -\overline{y}\right\| +\left\| y_{_{T}}\left( t_{T,2}\right) -\overline{y}\right\| \right] \\&\le K\left[ \left\| y_{_{T}}\left( t_{T,1}\right) -\overline{y}\right\| +\left\| y_{_{T}}\left( t_{T,2}\right) -\overline{y}\right\| \right] , \end{aligned}$$

where K is independent of T.

Then, by Lemma A.1, for any $s\in \left[ t_{T,1},t_{T,2}\right]$,

$$\begin{aligned} \Vert y_{_{T}}(s){-}\overline{y}\Vert ^2&{\le } K\left\{ \Vert y_{_{T}} \left( t_{T,1}\right) {-}\overline{y}\Vert ^2{+}\int _{t_{T,1}}^{t_{T,2}} \left[ \Vert u_{_{T}}(s){-}\overline{u}\Vert ^2 {+} \Vert C \,\left( y_{_{T}}(s) {-}\overline{y}\right) \Vert ^2\right] \hbox {d}s\right\} \\&{\le } K\left[ \left\| y_{_{T}}\left( t_{T,1}\right) -\overline{y}\right\| +\left\| y_{_{T}}\left( t_{T,2}\right) -\overline{y}\right\| \right] \le K\eta , \end{aligned}$$

Finally, for any $\varepsilon >0$, setting $\eta =\frac{\varepsilon ^2}{K}$ and $\tau (A,B,C,U,x,z,g,\varepsilon ) =\zeta (A,B,C,U,x,z,g,\eta )$, we get the thesis. $\square$

Appendix B. Riccati theory and proof of Proposition 1.5

Although the proofs of our main results (Theorems 1.2 and 1.3) do not rely on the use of the classical Riccati theory, which is not applicable to our case due to the constraints on the control, we note that in the unconstrained case $U={\mathbb {R}}^m$, we may use the Riccati theory to obtain the value function W(x) explicitly as a positively definite quadratic form. We recall that, following Theorem 1.3, the value function W(x) is the limiting profile of the asymptotic decomposition of the value function V(T, x).

The proof of Proposition 1.5 is based on the following well-known lemma, concerning the properties of the algebraic Riccati equation, and the corresponding Hamiltonian matrix

$$\begin{aligned} \text{ Ham }:=\begin{bmatrix} A&{}-BB^*\\ -C^*C&{}-A^* \end{bmatrix}. \end{aligned}$$

One can realize that $\text{ Ham }$ is the associated matrix to the optimality system (B.5).

Lemma A.1

Assume (A, B) is stabilizable and (A, C) is detectable. Then,

• (1) there exists a unique symmetric positive semidefinite solution to the Algebraic Riccati Equation

  $$\begin{aligned} -\widehat{E}A-A^*\widehat{E}+\widehat{E}BB^*\widehat{E}=C^*C \qquad \text{(ARE) } \end{aligned}$$

  (B.1)

  such that $A-BB^*\widehat{E}$ is stable, i.e., the real part of the spectrum $\text{ Re }(\sigma (A-BB^*\widehat{E}))\subset (-\infty ,0)$;

• (2) set

  $$\begin{aligned} \Lambda :=\begin{bmatrix} I_n&{}S\\ \widehat{E}&{}\widehat{E}S+I_n \end{bmatrix}, \end{aligned}$$

  (B.2)

  where S is solution to the Lyapunov equation

  $$\begin{aligned} S(A-BB^*\widehat{E})^*+(A-BB^*\widehat{E})S=BB^*. \end{aligned}$$

  Then, $\Lambda$ is invertible and

  $$\begin{aligned} \Lambda ^{-1}\text{ Ham } \ \Lambda =\begin{bmatrix} A-BB^*\widehat{E}&{}0\\ 0&{}-(A-BB^*\widehat{E})^*. \end{bmatrix} \end{aligned}$$

  As a consequence, $\text{ Ham }$ is invertible and its spectrum does not intersect the imaginary axis.

The first part of the above lemma is Riccati theory (see, for instance, [15,  Fact 1-(a) and Fact 1-(f)] or [1]). The second part³ is taken from [43,  subsection III.B]. We are now ready to prove Proposition 1.5.

Proof of Proposition 1.5

First of all, let us show that the minimization of $J_{\infty ,x}$ is equivalent to the minimization of

$$\begin{aligned} \begin{array}{cccl} \widehat{J}_{\infty ,x}: &{} L^2_{\mathrm{loc}}(0,+\infty ;{\mathbb {R}}^m) &{} \longrightarrow &{} {\mathbb {R}}\cup \left\{ +\infty \right\} \\ &{} u &{} \longmapsto &{} \displaystyle \dfrac{1}{2} \int _0^{\infty } \left[ \Vert u(s)-\overline{u}\Vert ^2 + \Vert C \, \left( y(s) -\overline{y}\right) \Vert ^2\right] \hbox {d}s, \end{array} \end{aligned}$$

where y is the solution to (1.1), with initial datum x and control u. Let us show this by proceeding as in the proof of (2.3), and concluding with Lemma 2.4. We first consider the finite horizon cost functional with final cost $g=0$, that is

$$\begin{aligned} J_{T,x}(u)= & {} \dfrac{1}{2} \int _0^T \left[ \Vert u(s) -\overline{u}+\overline{u}\Vert ^2 + \Vert C \, y (s) -C\overline{y}+C\overline{y}-z\Vert ^2\right] \hbox {d}s \nonumber \\= & {} \dfrac{T}{2} \left[ \Vert \overline{u}\Vert ^2 +\Vert C\, \overline{y}-z\Vert ^2\right] +\dfrac{1}{2} \int _0^T \left[ \Vert u(s)-\overline{u}\Vert ^2 + \Vert C \, \left( y (s) -\overline{y}\right) \Vert ^2\right] \hbox {d}s\nonumber \\&+\int _0^T \left[ \left( \overline{u},u(s) -\overline{u}\right) _{{\mathbb {R}}^m} +\left( C\overline{y}-z,\, C \left( y (s) -\overline{y}\right) \right) _{{\mathbb {R}}^n}\right] \hbox {d}s\nonumber \\= & {} T\, V_s +\dfrac{1}{2} \int _0^T \left[ \Vert u(s)-\overline{u}\Vert ^2 + \Vert C \, \left( y (s)-\overline{y}\right) \Vert ^2\right] \hbox {d}s\nonumber \\&+\int _0^T \left[ \left( \overline{u},u(s) -\overline{u}\right) _{{\mathbb {R}}^m} +\left( C\overline{y}-z,\, C \left( y(s) -\overline{y}\right) \right) _{{\mathbb {R}}^n}\right] \hbox {d}s. \end{aligned}$$

(B.3)

Hence, one has

$$\begin{aligned}&\frac{1}{2}\int _{0}^{T}\left[ \Vert u(s)\Vert ^{2} + \Vert C\,y(s) -z\Vert ^{2} - V_{s}\right] \,\hbox {d}s = \dfrac{1}{2} \int _0^T \left[ \Vert u(s)-\overline{u}\Vert ^2 + \Vert C \, \left( y (s) -\overline{y}\right) \Vert ^2\right] \hbox {d}s\\&\qquad +\int _0^T \left[ \left( \overline{u},u(s) -\overline{u}\right) _{{\mathbb {R}}^m}+\left( C\overline{y} -z,\, C \left( y (s)-\overline{y}\right) \right) _{{\mathbb {R}}^n}\right] \hbox {d}s. \end{aligned}$$

Then we focus on the term

$$\begin{aligned} \int _0^T\left( C\, \overline{y}-z, \,C \left( y(s)-\overline{y}\right) \right) _{{\mathbb {R}}^n}\hbox {d}s. \end{aligned}$$

(B.4)

We recall that the pair $(\overline{u},\overline{y})$ satisfies the steady optimality system which reads as

$$\begin{aligned} {\left\{ \begin{array}{ll} 0=A\overline{y}-BB^*\overline{p}\\ 0=A^*\overline{p}+C^*(C\, \overline{y}-z), \end{array}\right. } \end{aligned}$$

(B.5)

with $\overline{u}=-B^* \overline{p}$. On the other hand, the pairs $(u(\cdot ),y(\cdot ))$ and $(\overline{u},\overline{y})$ satisfy the equation in (1.1). Hence, we have

$$\begin{aligned} {\left\{ \begin{array}{ll} \frac{\hbox {d}}{\hbox {d}s}(y-\overline{y})=A(y-\overline{y}) +B(u-\overline{u})\qquad &{} s\in (0,T)\\ y(0)-\overline{y}=x-\overline{y}. \end{array}\right. } \end{aligned}$$

(B.6)

Then, using (B.5) and (B.6) and taking into account that $y(0)=x$ and $\overline{u}=-B^* \overline{p}$, we can compute the term (B.4) as follows:

$$\begin{aligned} \int _0^T\left( C\overline{y}-z,\, C \left( y(s) -\overline{y}\right) \right) _{{\mathbb {R}}^n}\hbox {d}s&=\int _0^T\left( C^*\left( C\overline{y}-z\right) , \, y(s)-\overline{y}\right) _{{\mathbb {R}}^n}\hbox {d}s\nonumber \\&=-\int _0^T\left( \overline{p},\, A\left( y(s) -\overline{y}\right) \right) _{{\mathbb {R}}^n}\hbox {d}s\nonumber \\&=-\int _0^T\left( \overline{p}, \frac{\hbox {d}}{\hbox {d}s} (y-\overline{y})-B(u-\overline{u})\right) _{{\mathbb {R}}^n}\hbox {d}s\nonumber \\&=\left( \overline{p},y(0)-\overline{y}\right) _{{\mathbb {R}}^n} -\left( \overline{p},y(T)-\overline{y}\right) _{{\mathbb {R}}^n}\nonumber \\&\quad + \int _0^T \left( B^*\overline{p},u(s) -\overline{u}\right) _{{\mathbb {R}}^m} \hbox {d}s\nonumber \\&=\left( \overline{p},x-y(T)\right) _{{\mathbb {R}}^n} -\int _0^T \left( \overline{u},u(s) -\overline{u}\right) _{{\mathbb {R}}^m} \hbox {d}s. \end{aligned}$$

(B.7)

Finally, the conclusion follows by combining (B.3) and (B.7) and then letting $T\rightarrow +\infty$ since from Lemma 2.4 one has $u-\overline{u}\in L^{2}(0,+\infty ;{\mathbb {R}}^{m})$, $y-\overline{y}\in L^{2}(0,+\infty ;{\mathbb {R}}^{n})$ and $y(T) \rightarrow \overline{y}$.

By [34, Theorem 3.7 pages 237-238], there exists a unique minimizer $u^{*}$ for $\widehat{J}_{\infty ,x}$, given by (1.17) and

$$\begin{aligned} \inf _{L^2_{\mathrm{loc}}(0,+\infty ;{\mathbb {R}}^m)} \widehat{J}_{\infty ,x}(u)=\dfrac{1}{2} \left( x-\overline{y}\right) ^*\widehat{E}\left( x-\overline{y}\right) , \end{aligned}$$

whence, by (1.10) and (2.15) which is now an equality,

$$\begin{aligned} W(x)= & {} \inf _{L^2_{\mathrm{loc}}(0,+\infty ;{\mathbb {R}}^m)} \widehat{J}_{\infty ,x}(u)+\left( \overline{p},x -\overline{y}\right) _{{\mathbb {R}}^n}\\= & {} \dfrac{1}{2} \left( x-\overline{y}\right) ^*\widehat{E} \left( x-\overline{y}\right) +(\overline{p},x -\overline{y})_{{\mathbb {R}}^n}, \end{aligned}$$

as desired. $\square$