The nonconvex and nonsmooth finite-sum optimization problem with linear constraint has attracted much attention in the fields of artificial intelligence, computer, and mathematics, due to its wide applications in machine learning and the lack of efficient algorithms with convincing convergence theories. A popular approach to solve it is the stochastic Alternating Direction Method of Multipliers (ADMM), but most stochastic ADMM-type methods focus on convex models. In addition, the variance reduction (VR) and acceleration techniques are useful tools in the development of stochastic methods due to their simplicity and practicability in providing acceleration characteristics of various machine learning models. However, it remains unclear whether accelerated SVRG-ADMM algorithm (ASVRG-ADMM), which extends SVRG-ADMM by incorporating momentum techniques, exhibits a comparable acceleration characteristic or convergence rate in the nonconvex setting. To fill this gap, we consider a general nonconvex nonsmooth optimization problem and study the convergence of ASVRG-ADMM. By utilizing a well-defined potential energy function, we establish its sublinear convergence rate , where denotes the iteration number. Furthermore, under the additional Kurdyka–Lojasiewicz (KL) property which is less stringent than the frequently used conditions for showcasing linear convergence rates, such as strong convexity, we show that the ASVRG-ADMM sequence almost surely has a finite length and converges to a stationary solution with a linear convergence rate. Several experiments on solving the graph-guided fused lasso problem and regularized logistic regression problem validate that the proposed ASVRG-ADMM performs better than the state-of-the-art methods.

中文翻译：

---

用于非凸非光滑有限和优化的加速随机 ADMM

具有线性约束的非凸非光滑有限和优化问题因其在机器学习中的广泛应用以及缺乏具有令人信服的收敛理论的高效算法而在人工智能、计算机和数学领域引起了广泛关注。解决该问题的一种流行方法是随机交替方向乘子法 (ADMM)，但大多数随机 ADMM 类型的方法都专注于凸模型。此外，方差减少（VR）和加速技术由于其在提供各种机器学习模型的加速特性方面的简单性和实用性，是随机方法开发中的有用工具。然而，目前尚不清楚通过结合动量技术扩展 SVRG-ADMM 的加速 SVRG-ADMM 算法（ASVRG-ADMM）是否在非凸设置中表现出可比的加速特性或收敛速度。为了填补这一空白，我们考虑一般的非凸非光滑优化问题并研究 ASVRG-ADMM 的收敛性。通过利用明确定义的势能函数，我们建立了其次线性收敛率 ，其中 表示迭代次数。此外，在额外的 Kurdyka-Lojasiewicz (KL) 属性下，该属性比展示线性收敛率的常用条件（例如强凸性）不太严格，我们表明 ASVRG-ADMM 序列几乎肯定具有有限长度并收敛到具有线性收敛速度的平稳解。解决图引导融合套索问题和正则化逻辑回归问题的几个实验验证了所提出的 ASVRG-ADMM 的性能优于最先进的方法。