This study investigates the integration of two prominent neural network representations into a hybrid cognitive model for solving a natural language task, where pre-trained large-language models serve as global learners and recurrent neural networks offer more “local” task-specific representations in the neural network. To explore the fusion of these two types of representations, we employ an autoencoder to transform them between each other or fuse them into a single model. Our exploration identifies a computational constraint, which we term limited diffusibility, highlighting the limitations of hybrid systems that operate with distinct types of representation. The findings from our hybrid system confirm the crucial role of global knowledge in adapting to a new learning task, as having only local knowledge greatly reduces the system’s transferability.

本研究研究了将两种著名的神经网络表示集成到解决自然语言任务的混合认知模型中，其中预先训练的大语言模型充当全局学习器，而循环神经网络在自然语言任务中提供更多“本地”特定于任务的表示。神经网络。为了探索这两种类型表示的融合，我们使用自动编码器来将它们相互转换或将它们融合成一个模型。我们的探索确定了计算约束，我们将其称为有限扩散性，突出了使用不同类型表示的混合系统的局限性。我们的混合系统的研究结果证实了全局知识在适应新学习任务中的关键作用，因为只有局部知识会大大降低系统的可迁移性。