Reinforcement learning (RL) engages a network of areas, including the orbitofrontal cortex (OFC), ventral striatum (VS), amygdala (AMY), and mediodorsal thalamus (MDt). This study examined RL mediated by gains and losses of symbolic reinforcers across this network. Monkeys learned to select options that led to gaining tokens and avoid options that led to losing tokens. Tokens were cashed out for juice rewards periodically. OFC played a dominant role in coding information about token updates, suggesting that the cortex is more important than subcortical structures when learning from symbolic outcomes. We also found that VS showed increased responses specific to appetitive outcomes, and AMY responded to the salience of outcomes. In addition, analysis of network activity showed that symbolic reinforcement was calculated by temporal differentiation of accumulated tokens. This process was mediated by dynamics within the OFC-MDt-VS circuit. Thus, we provide a neurocomputational account of learning from symbolic gains and losses.

中文翻译：

---

从符号收益和损失中学习的网络计算

强化学习 (RL) 涉及多个区域网络，包括眶额皮质 (OFC)、腹侧纹状体 (VS)、杏仁核 (AMY) 和内侧丘脑 (MDt)。这项研究检验了由该网络中符号强化物的得失所介导的强化学习。猴子学会了选择导致获得代币的选项并避免导致失去代币的选项。代币会定期兑现以获得果汁奖励。 OFC 在编码有关令牌更新的信息中发挥了主导作用，这表明在从符号结果中学习时，皮层比皮层下结构更重要。我们还发现，VS 对食欲结果的特定反应有所增加，而 AMY 对结果的显着性做出了反应。此外，对网络活动的分析表明，符号强化是通过累积令牌的时间微分来计算的。这个过程是由 OFC-MDt-VS 电路内的动力学介导的。因此，我们提供了从符号增益和损失中学习的神经计算解释。