Abstract
We construct an importance sampling method for computing statistics related to rare events for weakly interacting diffusions. Standard Monte Carlo methods behave exponentially poorly with the number of particles in the system for such problems. Our scheme is based on subsolutions of a Hamilton–Jacobi–Bellman (HJB) equation on Wasserstein space which arises in the theory of mean-field (McKean–Vlasov) control. We identify conditions under which such a scheme is asymptotically optimal. In the process, we make connections between the large deviations principle for the empirical measure of weakly interacting diffusions, mean-field control, and the HJB equation on Wasserstein space. We also provide evidence, both analytical and numerical, that with sufficient regularity of the HJB equation, our scheme can have vanishingly small relative error in the many particle limit.
Similar content being viewed by others
References
Adams, S., Dirr, N., Peletier, M.A., Zimmer, J.: From a large-deviations principle to the Wasserstein gradient flow: a new micro-macro passage. Commun. Math. Phys. 307(3), 791–815 (2011)
Adams, S., Dirr, N., Peletier, M.A., Zimmer, J.: Large deviations and gradient flows. Philos. Trans. R. Soc. A 371(2005), 20120341 (2013)
Babin, V., Roland, C., Sagui, C.: Adaptively biased molecular dynamics for free energy calculations. J. Chem. Phys. 128(13), 128–135 (2008)
Backhoff, J., Conforti, G., Gentil, I., Léonard, C.: The mean field Schrödinger problem: ergodic behavior, entropy estimates and functional inequalities. Probab. Theory Relat. Fields 178(2), 475–530 (2020)
Basei, M., Pham, H.: A weak martingale approach to linear-quadratic McKean–Vlasov stochastic control problems. J. Optim. Theory Appl. 181(2), 347–382 (2019)
Bashiri, K., Menz, G.: Metastability in a continuous mean-field model at low temperature and strong interaction. Stoch. Process. Appl. 134, 132–173 (2021)
Ben Rached, N., Haji-Ali, A.-L., Pillai, S.M.S., Tempone, R.: Multilevel importance sampling for McKean–Vlasov stochastic differential equation. arXiv:2208.03225 [math.NA] (2022)
Ben Rached, N., Haji-Ali, A.-L., Pillai, S.M.S., Tempone, R.: Single level importance sampling for McKean–Vlasov stochastic differential equations. arXiv: 2207.06926 [math.NA] (2022)
Bezemek, Z.W., Spiliopoulos, K.: Large deviations for interacting multiscale particle systems. Stoch. Process. Appl. 155, 27–108 (2023)
Bezemek, Z.W., Spiliopoulos, K.: Moderate deviations for fully coupled multiscale weakly interacting particle systems. Stochastics and Partial Differential Equations: Analysis and Computations (2023). https://doi.org/10.1007/s40072-023-00301-0
Borrell, E.R., Quer, J., Richter, L., Schütte, C.: Improving control based importance sampling strategies for metastable diffusions via adapted metadynamics. SIAM J. Sci. Comput. (2023). https://doi.org/10.1137/22M1503464
Bouchet, F., Gawȩdzki, K., Nardini, C.: Perturbative calculation of quasi-potential in non-equilibrium diffusions: a mean-field example. J. Stat. Phys. 163, 1157–1210 (2016)
Boué, M., Dupuis, P.: A variational representation for certain functionals of Brownian motion. Ann. Probab. 26(4), 1641–1659 (1998)
Budhiraja, A., Conroy, M.: Asymptotic behavior of stochastic currents under large deviation scaling with mean field interaction and vanishing noise. Annali della Scuola Normale di Pisa - Classe di Scienze 23(5), 1749–1805 (2022)
Budhiraja, A., Conroy, M.: Empirical measure and small noise asymptotics under large deviation scaling for interacting diffusions. J. Theor. Probab. 35, 295–349 (2022)
Budhiraja, A., Dupuis, P., Fischer, M.: Large deviation properties of weakly interacting particles via weak convergence methods. Ann. Probab. 40(1), 74–100 (2012)
Budhiraja, A., Wu, R.: Moderate deviation principles for weakly interacting particle systems. Probab. Theory Relat. Fields 168, 721–771 (2017)
Burzoni, M., Ignazio, V., Reppen, A.M., Soner, H.M.: Viscosity solutions for controlled McKean–Vlasov jump-diffusions. SIAM J. Control Optim. 58(3), 1676–1699 (2020)
Carmona, R., Delarue, F.: Probabilistic Theory of Mean Field Games with Applications I. Springer, New York (2018)
Carmona, R., Laurière, M.: Convergence analysis of machine learning algorithms for the numerical solution of mean field control and games: II—the finite horizon case. Ann. Appl. Probab. 32(6), 4065–4105 (2022)
Chassagneux, J.F., Crisan, D., Delarue, F.: A probabilistic approach to classical solutions of the master equation for large population equilibria. Memoirs of the American Mathematical Society (2022) 280, 1379. https://doi.org/10.1090/memo/1379
Chassagneux, J.F., Crisan, D., Delarue, F.: Numerical method for FBSDEs of McKean–Vlasov type. Ann. Appl. Probab. 29(3), 1640–1684 (2019)
Chaudru de Raynal, P.E., Frikha, N.: Well-posedness for some non-linear diffusion processes and related PDE on the Wasserstein space. Journal de Mathematiques Pures et Appliquees 159, 1–167 (2022)
Chaintron, L.P., Diez, A.: Propagation of chaos: a review of models, methods and applications. I. Models and methods. Kinet. Relat. Models 15(6), 895–1015 (2022)
Comer, J., Gumbart, J.C., Hénin, J., Lelièvre, T., Pohorille, A., Chipot, C.: The adaptive biasing force method: everything you always wanted to know but were afraid to ask. J. Phys. Chem. B 118(3), 1129–1151 (2015)
Cosso, A., Gozzi, F., Kharroubi, I., Pham, H., Rosestolato, M.: Master Bellman equation in the Wasserstein space: uniqueness of viscosity solutions. Trans. Am. Math. Soc. (2023). https://doi.org/10.1090/tran/8986
Cosso, A., Gozzi, F., Kharroubi, I., Pham, H., Rosestolato, M.: Optimal control of path-dependent McKean–Vlasov SDEs in infinite dimension. Ann. Appl. Probab. 33(4), 2863–2918 (2023)
Dawson, D.A.: Critical dynamics and fluctuations for a mean-field model of cooperative behavior. J. Stat. Phys. 31, 29–85 (1983)
Dawson, D.A., Gärtner, J.: Large deviations from the Mckean–Vlasov limit for weakly interacting diffusions. Stochastics 20(4), 247–308 (1987)
Del Moral, P., Guionnet, A.: Large deviations for interacting particle systems: applications to non-linear filtering. Stoch. Process. Appl. 78(1), 69–95 (1998)
Delarue, F., Lacker, D., Ramanan, K.: From the master equation to mean field game limit theory: large deviations and concentration of measure. Ann. Probab. 48(1), 211–263 (2020)
dos Reis, G., Salkeld, W., Tugaut, J.: Freidlin–Wentzell LDP in path space for McKean–Vlasov equations and the functional iterated logarithm law. Ann. Probab. 29(3), 1487–1540 (2019)
dos Reis, G., Smith, G., Tankov, P.: Importance sampling for McKean–Vlasov SDEs. Appl. Math. Comput. (2023). https://doi.org/10.1016/j.amc.2023.128078
Dupuis, P., Ellis, R.S.: A Weak Convergence Approach to the Theory of Large Deviations. Wiley, New York (1997)
Dupuis, P., Spiliopoulos, K., Zhou, X.: Escaping from an attractor: importance sampling and rest points I. Ann. Appl. Probab. 25(5), 2909–2958 (2015)
Dupuis, P., Spiliopoulos, K., Wang, H.: Importance sampling for multiscale diffusions. Multiscale Model. Simul. 10(1), 1–27 (2012)
Dupuis, P., Wang, H.: Importance sampling, large deviations, and differential games. Stoch. Stoch. Rep. 76(6), 481–508 (2004)
Dupuis, P., Wang, H.: Subsolutions of an Isaacs equation and efficient schemes for importance sampling. Math. Oper. Res. 32(3), 723–757 (2007)
E, W., Ren, W., Vanden-Eijnden, E.: Minimum action method for the study of rare events. Commun. Pure Appl. Math. 57(5), 637–656 (2004)
Ebener, L., Margazoglou, G., Friedrich, J., Biferale, L., Grauer, R.: Instanton based importance sampling for rare events in stochastic PDEs. Chaos 29(6), 063102 (2019)
Feng, J., Katsoulakis, M.: A comparison principle for Hamilton–Jacobi equations related to controlled gradient flows in infinite dimensions. Arch. Ration. Mech. Anal. 192, 275–310 (2009)
Feng, J., Kurtz, G.: Large Deviations for Stochastic Processes. American Mathematical Society, Providence (2006)
Feng, J., Mikami, T., Zimmer, J.: A Hamilton–Jacobi PDE associated with hydrodynamic fluctuations from a nonlinear diffusion equation. Commun. Math. Phys. 385, 1–54 (2021)
Feng, J., Nguyen, T.: Hamilton–Jacobi equations in space of measures associated with a system of conservation laws. Journal de Mathématiques Pures et Appliquées 97(4), 318–390 (2012)
Fischer, M.: On the form of the large deviation rate function for the empirical measures of weakly interacting systems. Bernoulli 20(4), 1765–1801 (2014)
Fleming, W.H., James, M.R.: Asymptotic series and exit time probabilities. Ann. Probab. 20(3), 1369–1384 (1992)
Fouque, J.P., Zhang, Z.: Deep learning methods for mean field control problems with delay. Front. Appl. Math. Stat. 12, 1369–1384 (2020)
Freidlin, M.I., Wentzell, A.D.: Random Perturbations of Dynamical Systems. Springer, Heidelberg (2012)
Garnier, J., Papanicolaou, G., Yang, T.W.: Large deviations for a mean field model of systemic risk. SIAM J. Financ. Math. 4(1), 151–184 (2013)
Garnier, J., Papanicolaou, G., Yang, T.W.: Consensus convergence with stochastic effects. Vietnam J. Math. 45, 51–75 (2017)
Gasteratos, I., Salins, M., Spiliopoulos, K.: Importance sampling for stochastic reaction-diffusion equations in the moderate deviation regime. arXiv:2206.00646 [math.PR] (2022)
Gavish, N., Nyquist, P., Peletier, M.: Large deviations and gradient flows for the Brownian one-dimensional hard-rod system. Potential Anal. 58, 71–121 (2021)
Geha, M., Jacquier, A., Žurič, Ž.: Large and moderate deviations for importance sampling in the Heston model. SSRN Electronic Journal (2021). https://doi.org/10.2139/ssrn.3952672
Germain, M., Laurière, M., Pham, H., Warin, X.: DeepSets and their derivative networks for solving symmetric PDEs. J. Sci. Comput. 91, 63 (2022)
Germain, M., Mikael, J., Warin, X.: Numerical resolution of McKean–Vlasov FBSDEs using neural networks. Methodol. Comput. Appl. Probab. 24, 2557–2586 (2022)
Germain, M., Pham, H., Warin, X.: Rate of convergence for particle approximation of PDEs in Wasserstein space. J. Appl. Probab. 59(4), 992–1008 (2022)
Grafke, T., Schäfer, T., Vanden-Eijnden, E.: Sharp asymptotic estimates for expectations, probabilities, and mean first passage times in stochastic systems with small noise. Commun. Pure Appl. Math. (2023). https://doi.org/10.1002/cpa.22177 10
Grünbaum, F.A.: Propagation of chaos for the Boltzmann equation. Arch. Ration. Mech. Anal. 42, 323–345 (1971)
Gvalani, R.S., Schlichting, A.: Barriers of the McKean–Vlasov energy via a mountain pass theorem in the space of probability measures. J. Funct. Anal. 279(11), 437–461 (2020)
Han, J., Hu, R., Long, J.: Learning high-dimensional McKean–Vlasov forward-backward stochastic differential equations with general distribution dependence. arXiv:2204.11924 [math.OC] (2022)
Hartmann, C., Banisch, R., Sarich, M., Badowski, T., Schütte, C.: Characterization of rare events in molecular dynamics. Entropy 16(1), 350–376 (2014)
Hartmann, C., Richter, L.: Nonasymptotic bounds for suboptimal importance sampling. arXiv: 2102.09606 [math.ST] (2021)
Kac, M.: Probability and Related Topics in Physical Sciences. American Mathematical Society, Providence (1957)
Karatzas, I., Shreve, S.: Brownian Motion and Stochastic Calculus. Springer, New York (1998)
Lacker, D.: Limit theory for controlled McKean–Vlasov dynamics. SIAM J. Control Optim. 55(3), 1641–1672 (2017)
Laurière, M.: Numerical methods for mean field games and mean field type control. Proc. Symp. Appl. Math. 78, 221–282 (2021)
Laurièr, M., Pironneau, O.: Dynamic programming for mean-field type control. J. Optim. Theory Appl. 169, 902–924 (2016)
Liu, W., Song, Y., Zhai, J., Zhang, T.: Large and moderate deviation principles for McKean–Vlasov SDEs with jumps. Potential Anal. 59(3), 1141–1190 (2023)
Morse, M.R., Spiliopoulos, K.: Importance sampling for slow-fast diffusions based on moderate deviations. SIAM J. Multiscale Model. Simul. 18(1), 315–350 (2020)
Nilsson, V., Nyquist, P.: A note on large deviations for interacting particle dynamics for finding mixed equilibria in zero-sum games. arXiv:2206.15177 [stat.ML] (2022)
Nüsken, N., Richter, L.: Solving high-dimensional Hamilton–Jacobi–Bellman PDEs using neural networks: perspectives from the theory of controlled diffusions and measures on path space. Part. Differ. Equ. Appl. 2, 48 (2021)
Orrieri, C.: Large deviations for interacting particle systems: joint mean-field and small-noise limit. Electron. J. Probab. 25, 1–44 (2020)
Pham, H., Wei, X.: Bellman equation and viscosity solutions for mean-field stochastic control problem. ESAIM: Control Optim. Calc. Var. 24(1), 437–461 (2018)
Salins, M., Spiliopoulos, K.: Rare event simulation via importance sampling for linear SPDE’s. Stoch. Part. Differ. Equ.: Anal. Comput. 5(4), 652–690 (2017)
Spiliopoulos, K.: Nonasymptotic performance analysis of importance sampling schemes for small noise diffusions. J. Appl. Probab. 53(3), 797–810 (2015)
Talbi, M., Touzi, N., Zhang, J.: Dynamic programming equation for the mean field optimal stopping problem. SIAM J. Control Optim. 61(4), 2140–2164 (2023)
Tong, S., Stadler, G.: Large deviation theory-based adaptive importance sampling for rare events in high dimensions. SIAM/ASA J. Uncertain. Quant. 11(3), 788–813 (2023)
Valsson, O., Tiwary, P., Parrinello, M.: Enhancing important fluctuations: rare events and metadynamics from a conceptual viewpoint. Annu. Rev. Phys. Chem. 67, 159–184 (2016)
Vanden-Eijnden, E., Weare, J.: Rare event simulation of small noise diffusions. Commun. Pure Appl. Math. 65(12), 1770–1803 (2012)
Yong, J.: Linear-quadratic optimal control problems for mean-field stochastic differential equations. SIAM J. Control Optim. 51(4), 2809–2838 (2013)
Acknowledgements
Bezemek was partially supported by NSF-DMS 2107856. Heldman was partially supported by NSF-DMS 1902854, ARO W911NF-20-1-0244, and a subgrant of NSF-OAC 2139536. The funding agencies are not expected to gain or lose financially through publication of this manuscript. The authors have no relevant financial interests to disclose.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing Interests
The authors have not disclosed any competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Z.W.B. was partially supported by the National Science Foundation (DMS 2107856). The majority of this work was completed while Z.W.B. was a PhD candidate at Boston University. M.H. was partially supported by NSF-DMS 1902854, ARO W911NF-20-1-0244, and a subgrant of NSF-OAC 2139536. The authors of the paper would like to thank both reviewers for a their careful and constructive reviews of this article.
Appendix A. Differentiation on Spaces of Measures
Appendix A. Differentiation on Spaces of Measures
Definition A.1
Given a function \(u:\mathcal { P}_2(\mathbb {R}^d)\rightarrow \mathbb {R}\), we may define a lifting of u to \(\tilde{u}:L^2({\tilde{\Omega }},{\tilde{\mathcal {F}}},{\tilde{\mathbb {P}}};\mathbb {R}^d)\rightarrow \mathbb {R}\) via \({\tilde{u}} (X) = u(\mathcal { L}(X))\) for \(X\in L^2({\tilde{\Omega }},{\tilde{\mathcal {F}}},{\tilde{\mathbb {P}}};\mathbb {R}^d)\). Here we assume \({\tilde{\Omega }}\) is a Polish space, \({\tilde{\mathcal {F}}}\) its Borel \(\sigma \)-field, and \({\tilde{\mathbb {P}}}\) is an atomless probability measure (since \({\tilde{\Omega }}\) is Polish, this is equivalent to every singleton having zero measure).
Here:
\(\mathcal { P}_2(\mathbb {R}^d)\) is a Polish space under the \(L^2\)-Wasserstein distance
where \(\mathcal { C}_{\mu _1,\mu _2}\) denotes the set of all probability measures on \(\mathbb {R}^{d}\times \mathbb {R}^d\) with first marginal \(\mu _1\) and second marginal \(\mu _2\).
We say u is L-differentiable or Lions-differentiable at \(\mu _0\in \mathcal { P}_2(\mathbb {R}^d)\) if there exists a random variable \(X_0\) on some \(({\tilde{\Omega }},{\tilde{\mathcal {F}}},{\tilde{\mathbb {P}}})\) satisfying the above assumptions such that \(\mathcal { L}(X_0)=\mu _0\) and \({\tilde{u}}\) is Fréchet differentiable at \(X_0\).
The Fréchet derivative of \({\tilde{u}}\) can be viewed as an element of \(L^2({\tilde{\Omega }},{\tilde{\mathcal {F}}},{\tilde{\mathbb {P}}};\mathbb {R}^d)\) by identifying \(L^2({\tilde{\Omega }},{\tilde{\mathcal {F}}},{\tilde{\mathbb {P}}};\mathbb {R}^d)\) and its dual. From this, one can find that if u is L-differentiable at \(\mu _0\in \mathcal { P}_2(\mathbb {R}^d)\), there is a deterministic measurable function \(\xi : \mathbb {R}^d\rightarrow \mathbb {R}^d\) such that \(D\tilde{u}(X_0)=\xi (X_0)\), and that \(\xi \) is uniquely defined \(\mu _0\)-almost everywhere on \(\mathbb {R}^d\). We denote this equivalence class of \(\xi \in L^2(\mathbb {R}^d,\mu _0;\mathbb {R}^d)\) by \(\partial _\mu u(\mu _0)\) and call \(\partial _\mu u(\mu _0)[\cdot ]:\mathbb {R}^d\rightarrow \mathbb {R}^d\) the Lions derivative of u at \(\mu _0\). Note that this definition is independent of the choice of \(X_0\) and \(({\tilde{\Omega }},{\tilde{\mathcal {F}}},{\tilde{\mathbb {P}}})\). See [19] Section 5.2.
To avoid confusion when u depends on more variables than just \(\mu \), if \(\partial _\mu u(\mu _0)\) is differentiable at \(z_0\in \mathbb {R}^d\), we denote its derivative at \(z_0\) by \(\partial _z\partial _\mu u(\mu _0)[z_0]\).
Definition A.2
([19] Definition 5.83) We say \(u:\mathcal { P}_2(\mathbb {R}^d)\rightarrow \mathbb {R}\) is Fully \(\mathbf {C^2}\) if the following conditions are satisfied:
-
(1)
u is \(C^1\) in the sense of L-differentiation, and its first derivative has a jointly continuous version \(\mathcal { P}_2(\mathbb {R}^d)\times \mathbb {R}^d\ni (\mu ,z)\mapsto \partial _\mu u(\mu )[z]\in \mathbb {R}^d\).
-
(2)
For each fixed \(\mu \in \mathcal { P}_2(\mathbb {R}^d)\), the version of \(\mathbb {R}^d\ni z\mapsto \partial _\mu u(\mu )[z]\in \mathbb {R}^d\) from the first condition is differentiable on \(\mathbb {R}^d\) in the classical sense and its derivative is given by a jointly continuous function \(\mathcal { P}_2(\mathbb {R}^d)\times \mathbb {R}^d\ni (\mu ,z)\mapsto \partial _z\partial _\mu u(\mu )[z]\in \mathbb {R}^{d\times d}\).
-
(3)
For each fixed \(z\in \mathbb {R}^d\), the version of \(\mathcal { P}_2(\mathbb {R}^d)\ni \mu \mapsto \partial _\mu u(\mu )[z]\in \mathbb {R}^d\) in the first condition is continuously L-differentiable component-by-component, with a derivative given by a function \(\mathcal { P}_2(\mathbb {R}^d)\times \mathbb {R}^d\times \mathbb {R}^d\ni (\mu ,z,z')\mapsto \partial ^2_\mu u(\mu )[z][z']\in \mathbb {R}^{d\times d}\) such that for any \(\mu \in \mathcal { P}_2(\mathbb {R}^d)\) and \(X\in L^2({\tilde{\Omega }},{\tilde{\mathcal {F}}},{\tilde{\mathbb {P}}};\mathbb {R}^d)\) with \(\mathcal { L}(X)=\mu \), \(\partial ^2_{\mu }u(\mu )[z][X]\) gives the Fréchet derivative at X of \(L^2({\tilde{\Omega }},{\tilde{\mathcal {F}}},{\tilde{\mathbb {P}}};\mathbb {R}^d)\ni X'\mapsto \partial _\mu u(\mathcal { L}(X'))[z]\) for every \(z\in \mathbb {R}^d\). Denoting \(\partial ^2_\mu u(\mu )[z][z']\) by \(\partial ^2_\mu u(\mu )[z,z']\), the map \(\mathcal { P}_2(\mathbb {R}^d)\times \mathbb {R}^d\times \mathbb {R}^d\ni (\mu ,z,z')\mapsto \partial ^2_\mu u(\mu )[z,z']\) is also assumed to be continuous in the product topology.
We recall now a useful connection between the Lions derivative as defined in A.1 and the empirical measure.
Proposition A.3
For \(g:\mathcal { P}_2(\mathbb {R}^d)\rightarrow \mathbb {R}\) which is fully \(C^2\) in the sense of definition A.2, we can define the empirical projection of g, as \(g^N: (\mathbb {R}^d)^N\rightarrow \mathbb {R}\) given by
Then \(g^N\) is twice differentiable on \((\mathbb {R}^d)^N\), and for each \(x_1,..,x_N\in \mathbb {R}^d\), \((i,j)\in \lbrace 1,...,N \rbrace ^2\):
and
Proof
This follows from Propositions 5.35 and 5.91 of [19]. \(\square \)
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Bezemek, Z.W., Heldman, M. Importance Sampling for the Empirical Measure of Weakly Interacting Diffusions. Appl Math Optim 89, 7 (2024). https://doi.org/10.1007/s00245-023-10075-5
Accepted:
Published:
DOI: https://doi.org/10.1007/s00245-023-10075-5