We and other animals learn because there is some aspect of the world about which we are uncertain. This uncertainty arises from initial ignorance, and from changes in the world that we do not perfectly know; the uncertainty often becomes evident when our predictions about the world are found to be erroneous. The Rescorla-Wagner learning rule, which specifies one way that prediction errors can occasion learning, has been hugely influential as a characterization of Pavlovian conditioning and, through its equivalence to the delta rule in engineering, in a much wider class of learning problems. Here, we review the embedding of the Rescorla-Wagner rule in a Bayesian context that is precise about the link between uncertainty and learning, and thereby discuss extensions to such suggestions as the Kalman filter, structure learning, and beyond, that collectively encompass a wider range of uncertainties and accommodate a wider assortment of phenomena in conditioning.

中文翻译：

---

贝叶斯强化学习：基本概述

我们和其他动物之所以学习，是因为世界的某些方面我们是不确定的。这种不确定性源于最初的无知，源于我们并不完全了解世界的变化；当我们对世界的预测被发现是错误的时，不确定性往往会变得明显。 Rescorla-Wagner 学习规则指定了预测错误引发学习的一种方式，作为巴甫洛夫条件反射的表征，并且通过其与工程中的 Delta 规则的等价性，在更广泛的学习问题中发挥了巨大的影响力。在这里，我们回顾了 Rescorla-Wagner 规则在贝叶斯环境中的嵌入，该规则精确地描述了不确定性和学习之间的联系，从而讨论了卡尔曼滤波器、结构学习等建议的扩展，这些建议共同涵盖了更广泛的领域。范围的不确定性并适应调节中更广泛的现象。