A key aspect of cognitive flexibility is to efficiently make use of earlier experience to attain one’s goals. This requires learning, but also a modular, and more specifically hierarchical, structure. We hold that both are required, but combining them leads to several computational challenges that brains and artificial agents (learn to) deal with. In a hierarchical structure, meta-decisions must be made, of which two types can be distinguished. First, a (meta-)decision may involve choosing which (lower-level) modules to select (module choice). Second, it may consist of choosing appropriate parameter settings within a module (parameter tuning). Furthermore, prediction error monitoring may allow determining the right meta-decision (module choice or parameter tuning). We discuss computational challenges and empirical evidence relative to how these two meta-decisions may be implemented to support learning for cognitive flexibility.

中文翻译：

---

强化学习和元决策

认知灵活性的一个关键方面是有效利用早期经验来实现自己的目标。这需要学习，但也需要模块化的，更具体地说是分层的结构。我们认为两者都是必需的，但将它们结合起来会导致大脑和人工智能体（学习）处理的一些计算挑战。在层次结构中，必须做出元决策，元决策可以分为两种类型。首先，（元）决策可能涉及选择要选择哪些（较低级别）模块（模块选择）。其次，它可能包括在模块内选择适当的参数设置（参数调整）。此外，预测误差监控可以允许确定正确的元决策（模块选择或参数调整）。我们讨论了与如何实施这两个元决策以支持认知灵活性学习相关的计算挑战和经验证据。