当前位置: X-MOL 学术Ann. Math. Artif. Intel. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
To raise or not to raise: the autonomous learning rate question
Annals of Mathematics and Artificial Intelligence ( IF 1.2 ) Pub Date : 2023-08-08 , DOI: 10.1007/s10472-023-09887-6
Xiaomeng Dong , Tao Tan , Michael Potter , Yun-Chan Tsai , Gaurav Kumar , V. Ratna Saripalli , Theodore Trafalis

There is a parameter ubiquitous throughout the deep learning world: learning rate. There is likewise a ubiquitous question: what should that learning rate be? The true answer to this question is often tedious and time consuming to obtain, and a great deal of arcane knowledge has accumulated in recent years over how to pick and modify learning rates to achieve optimal training performance. Moreover, the long hours spent carefully crafting the perfect learning rate can come to nothing the moment your network architecture, optimizer, dataset, or initial conditions change ever so slightly. But it need not be this way. We propose a new answer to the great learning rate question: the Autonomous Learning Rate Controller. Find it at https://github.com/fastestimator/ARC/tree/v2.0.



中文翻译:

加还是不加:自主学习率问题

在深度学习领域有一个普遍存在的参数:学习率。同样还有一个普遍存在的问题:学习率应该是多少?要获得这个问题的真正答案通常是乏味且耗时的,并且近年来在如何选择和修改学习率以实现最佳训练性能方面积累了大量晦涩的知识。此外,当您的网络架构、优化器、数据集或初始条件发生如此微小的变化时,花费大量时间精心设计完美的学习率可能会变得毫无意义。但事情不必是这样的。我们为学习率问题提出了一个新的答案:自主学习率控制器。可以在 https://github.com/fastestimator/ARC/tree/v2.0 找到它。

更新日期:2023-08-09
down
wechat
bug