``The right to be forgotten'' ensured by laws for user data privacy becomes increasingly important. Machine unlearning aims to efficiently remove the effect of certain data points on the trained model parameters so that it can be approximately the same as if one retrains the model from scratch. This work proposes stochastic gradient Langevin unlearning, the first unlearning framework based on noisy stochastic gradient descent (SGD) with privacy guarantees for approximate unlearning problems under convexity assumption. Our results show that mini-batch gradient updates provide a superior privacy-complexity trade-off compared to the full-batch counterpart. There are numerous algorithmic benefits of our unlearning approach, including complexity saving compared to retraining, and supporting sequential and batch unlearning. To examine the privacy-utility-complexity trade-off of our method, we conduct experiments on benchmark datasets compared against prior works. Our approach achieves a similar utility under the same privacy constraint while using $2\%$ and $10\%$ of the gradient computations compared with the state-of-the-art gradient-based approximate unlearning methods for mini-batch and full-batch settings, respectively.

中文翻译：

---

随机梯度 Langevin 遗忘

法律保障的用户数据隐私“被遗忘权”变得越来越重要。机器去学习的目的是有效地消除某些数据点对训练后的模型参数的影响，使其与从头开始重新训练模型大致相同。这项工作提出了随机梯度 Langevin 失学习，这是第一个基于噪声随机梯度下降（SGD）的失学习框架，并在凸性假设下为近似失学习问题提供隐私保证。我们的结果表明，与全批量梯度更新相比，小批量梯度更新提供了卓越的隐私复杂性权衡。我们的取消学习方法有许多算法优势，包括与再训练相比节省了复杂性，以及支持顺序和批量取消学习。为了检查我们的方法的隐私-实用性-复杂性权衡，我们对基准数据集进行了与之前的工作进行比较的实验。与最先进的基于梯度的小批量和全批量近似遗忘方法相比，我们的方法在相同的隐私约束下实现了类似的效用，同时使用 $2\%$ 和 $10\%$ 的梯度计算分别设置。