Recurrent neural networks (RNNs) are often used to model circuits in the brain and can solve a variety of difficult computational problems requiring memory, error correction, or selection (Hopfield, 1982; Maass et al., 2002; Maass, 2011). However, fully connected RNNs contrast structurally with their biological counterparts, which are extremely sparse (about 0.1%). Motivated by the neocortex, where neural connectivity is constrained by physical distance along cortical sheets and other synaptic wiring costs, we introduce locality masked RNNs (LM-RNNs) that use task-agnostic predetermined graphs with sparsity as low as 4%. We study LM-RNNs in a multitask learning setting relevant to cognitive systems neuroscience with a commonly used set of tasks, 20-Cog-tasks (Yang et al., 2019). We show through reductio ad absurdum that 20-Cog-tasks can be solved by a small pool of separated autapses that we can mechanistically analyze and understand. Thus, these tasks fall short of the goal of inducing complex recurrent dynamics and modular structure in RNNs. We next contribute a new cognitive multitask battery, Mod-Cog, consisting of up to 132 tasks that expands by about seven-fold the number of tasks and task complexity of 20-Cog-tasks. Importantly, while autapses can solve the simple 20-Cog-tasks, the expanded task set requires richer neural architectures and continuous attractor dynamics. On these tasks, we show that LM-RNNs with an optimal sparsity result in faster training and better data efficiency than fully connected networks.

循环神经网络 (RNN) 通常用于对大脑中的电路进行建模，可以解决各种需要记忆、纠错或选择的困难计算问题（Hopfield，1982；Maass 等人，2002；Maass，2011）。然而，完全连接的 RNN 在结构上与生物对应的 RNN 形成鲜明对比，后者极其稀疏（约 0.1%）。受新皮质的启发，其中神经连接受到沿皮质层的物理距离和其他突触连接成本的限制，我们引入了局部屏蔽 RNN (LM-RNN)，它使用与任务无关的预定图，稀疏度低至 4%。我们在与认知系统神经科学相关的多任务学习环境中研究 LM-RNN，其中包含一组常用的任务，即20-Cog 任务（Yang 等人，2019）。我们通过反证法证明，20-Cog 任务可以通过一小部分独立的自动执行来解决，我们可以机械地分析和理解这些自动执行。因此，这些任务没有达到在 RNN 中引入复杂的循环动力学和模块化结构的目标。接下来，我们贡献了一种新的认知多任务电池Mod-Cog ，它由多达 132 个任务组成，其任务数量和任务复杂性是20-Cog-task的七倍左右。重要的是，虽然自动程序可以解决简单的20-Cog-tasks，但扩展的任务集需要更丰富的神经架构和连续的吸引子动力学。在这些任务中，我们表明，与完全连接的网络相比，具有最佳稀疏性的 LM-RNN 可以实现更快的训练和更好的数据效率。