Abstract

This paper focuses on solving a subclass of stochastic nonconvex-nonconcave black box optimization problems with a saddle point that satisfy the Polyak–Łojasiewicz (PL) condition. To solve this problem, we provide the first (to our best knowledge) gradient-free algorithm. The proposed approach is based on applying a gradient approximation (kernel approximation) to an oracle-biased stochastic gradient descent algorithm. We present theoretical estimates that guarantee its global linear rate of convergence to the desired accuracy. The theoretical results are checked on a model example by comparison with an algorithm using Gaussian approximation.

中文翻译：

---

求解具有 Polyak–Łojasiewicz 条件的随机鞍优化问题的无梯度算法

摘要

本文重点解决具有满足 Polyak–Łojasiewicz (PL) 条件的鞍点的随机非凸非凹黑盒优化问题的子类。为了解决这个问题，我们提供了第一个（据我们所知）无梯度算法。所提出的方法基于将梯度近似（核近似）应用于预言机偏置随机梯度下降算法。我们提出了理论估计，保证其全局线性收敛速度达到所需的精度。通过与使用高斯近似的算法进行比较，在模型示例上检查了理论结果。