当前位置：
X-MOL 学术
›
SIAM J. Control Optim.
›
论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Primal-Dual Regression Approach for Markov Decision Processes with General State and Action Spaces
SIAM Journal on Control and Optimization ( IF 2.2 ) Pub Date : 2024-02-12 , DOI: 10.1137/22m1526010 Denis Belomestny 1 , John Schoenmakers 2
SIAM Journal on Control and Optimization ( IF 2.2 ) Pub Date : 2024-02-12 , DOI: 10.1137/22m1526010 Denis Belomestny 1 , John Schoenmakers 2
Affiliation
SIAM Journal on Control and Optimization, Volume 62, Issue 1, Page 650-679, February 2024.
Abstract. We develop a regression-based primal-dual martingale approach for solving discrete time, finite-horizon MDPs. The state and action spaces may be finite or infinite (but regular enough) subsets of Euclidean space. Consequently, our method allows for the construction of tight upper and lower-biased approximations of the value functions, providing precise estimates of the optimal policy. Importantly, we prove error bounds for the estimated duality gap featuring polynomial dependence on the time horizon. Additionally, we observe sublinear dependence of the stochastic part of the error on the cardinality/dimension of the state and action spaces. From a computational perspective, our proposed method is efficient. Unlike typical duality-based methods for optimal control problems in the literature, the Monte Carlo procedures involved here do not require nested simulations.
更新日期：2024-02-13
Abstract. We develop a regression-based primal-dual martingale approach for solving discrete time, finite-horizon MDPs. The state and action spaces may be finite or infinite (but regular enough) subsets of Euclidean space. Consequently, our method allows for the construction of tight upper and lower-biased approximations of the value functions, providing precise estimates of the optimal policy. Importantly, we prove error bounds for the estimated duality gap featuring polynomial dependence on the time horizon. Additionally, we observe sublinear dependence of the stochastic part of the error on the cardinality/dimension of the state and action spaces. From a computational perspective, our proposed method is efficient. Unlike typical duality-based methods for optimal control problems in the literature, the Monte Carlo procedures involved here do not require nested simulations.
京公网安备 11010802027423号