当前位置: X-MOL 学术J. ACM › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Faster high-accuracy log-concave sampling via algorithmic warm starts
Journal of the ACM ( IF 2.5 ) Pub Date : 2024-03-20 , DOI: 10.1145/3653446
Jason M. Altschuler 1 , Sinho Chewi 2
Affiliation  

It is a fundamental problem to understand the complexity of high-accuracy sampling from a strongly log-concave density π on \(\mathbb {R}^d \). Indeed, in practice, high-accuracy samplers such as the Metropolis-adjusted Langevin algorithm (MALA) remain the de facto gold standard; and in theory, via the proximal sampler reduction, it is understood that such samplers are key for sampling even beyond log-concavity (in particular, for sampling under isoperimetric assumptions). This paper improves the dimension dependence of this sampling problem to \(\widetilde{O}(d^{1/2}) \). The previous best result for MALA was \(\widetilde{O}(d) \). This closes the long line of work on the complexity of MALA, and moreover leads to state-of-the-art guarantees for high-accuracy sampling under strong log-concavity and beyond (thanks to the aforementioned reduction). Our starting point is that the complexity of MALA improves to \(\widetilde{O}(d^{1/2}) \), but only under a warm start (an initialization with constant Rényi divergence w.r.t. π). Previous algorithms for finding a warm start took O(d) time and thus dominated the computational effort of sampling. Our main technical contribution resolves this gap by establishing the first \(\widetilde{O}(d^{1/2}) \) Rényi mixing rates for the discretized underdamped Langevin diffusion. For this, we develop new differential-privacy-inspired techniques based on Rényi divergences with Orlicz–Wasserstein shifts, which allow us to sidestep longstanding challenges for proving fast convergence of hypocoercive differential equations.



中文翻译:

通过算法热启动实现更快的高精度对数凹采样

理解从 \(\mathbb {R}^d \) 上的强对数凹密度π进行高精度采样的复杂性是一个基本问题。事实上,在实践中,高精度采样器(例如 Metropolis 调整的 Langevin 算法 (MALA))仍然是事实上的黄金标准;理论上,通过近端采样器的减少,可以理解,这种采样器对于超出对数凹性的采样(特别是在等周假设下的采样)来说是关键。本文将这个采样问题的维度依赖性改进为 \(\widetilde{O}(d^{1/2}) \)。MALA 之前的最佳成绩是 \(\widetilde{O}(d) \)。这结束了关于 MALA 复杂性的漫长工作,此外,还为强对数凹性及其他情况下的高精度采样提供了最先进的保证(由于上述的减少)。我们的出发点是,MALA 的复杂度提高到 \(\widetilde{O}(d^{1/2}) \),但仅限于热启动(使用恒定 Rényi 散度 wrt π进行初始化)。以前用于寻找热启动的算法需要O ( d ) 时间,因此主导了采样的计算工作量。我们的主要技术贡献通过建立离散欠阻尼朗之万扩散的第一个 \(\widetilde{O}(d^{1/2}) \) Rényi 混合速率来解决这一差距。为此,我们开发了基于 Rényi 散度和 Orlicz-Wasserstein 位移的新的受微分隐私启发的技术,这使我们能够避开证明低强制微分方程快速收敛的长期挑战。

更新日期:2024-03-20
down
wechat
bug