Abstract
The functionals of dynamic systems of various types are optimized using modern methods of reinforcement learning. The linear resource allocation problem, as well as the optimal consumption problem and its stochastic modifications are considered. In the reinforcement learning strategy gradient methods are used.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
REFERENCES
M. Sewak, “Deterministic policy gradient and the DDPG: Deterministic-policy-gradient-based approaches,” in Deep Reinforcement Learning (Springer, 2019), pp. 173–184.
J. Schulman, “Trust region policy optimization,” 2015. https://arxiv.org/abs/1502.05477.
T. Haarnoja, “Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,” 2018. https://arxiv.org/abs/1801.01290.
S. Huang, “A2C is a special case of PPO,” 2022. https://arxiv.org/abs/2205.09123.
J. Schulman, “Proximal policy optimization algorithms,” 2017. https://arxiv.org/abs/1707.06347.
L. Zhang, “Penalized proximal policy optimization for safe reinforcement learning,” 2022. https://arxiv.org/abs/2205.11814.
X. Chen, “The sufficiency of off-policyness: PPO is insufficient according to an off-policy measure,” 2022. https://arxiv.org/abs/2205.10047.
A. Ghosh, “Provably efficient model-free constrained RL with linear function approximation,” 2022. https://arxiv.org/abs/2206.11889.
Z. Song, “Safe-FinRL: A low bias and variance deep reinforcement learning implementation for high-freq stock trading,” 2022. https://arxiv.org/abs/2206.05910.
M. Kaledin, “Variance reduction for policy-gradient methods via empirical variance minimization,” 2022. https://arxiv.org/abs/2206.06827.
Q. Luo, “Finite-time analysis of fully decentralized single-timescale actor-critic,” 2022. https://arxiv.org/abs/2206.05733.
A. Deka, “ARC—actor residual critic for adversarial imitation learning,” 2022. https://arxiv.org/abs/2206.02095.
V. I. Tsurkov, Dynamical High-Dimensional Problems (Nauka, Moscow, 1988) [in Russian].
L. A. Beklaryan, A. Yu. Flerova, and A. A. Zhukova, Optimal Control Problems: A Teaching Book (Nauka, Moscow, 2018) [in Russian].
B. Øksendal, Stochastic Differential Equations: An Introduction with Applications (Springer, Berlin, 2003; Mir, Moscow, 2003).
L. S. Pontryagin, The Maximum Principle in Optimal Control (Nauka, Moscow, 2004) [in Russian].
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
The authors declare that they have no conflicts of interest.
Additional information
Publisher’s Note.
Pleiades Publishing remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Semenov, S.S., Tsurkov, V.I. Reinforcement Learning for Model Problems of Optimal Control. J. Comput. Syst. Sci. Int. 62, 508–521 (2023). https://doi.org/10.1134/S1064230723030127
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1134/S1064230723030127