This survey presents a necessarily incomplete (and biased) overview of results at the intersection of arithmetic circuit complexity, structured matrices and deep learning. Recently there has been some research activity in replacing unstructured weight matrices in neural networks by structured ones (with the aim of reducing the size of the corresponding deep learning models). Most of this work has been experimental and in this survey, we formalize the research question and show how a recent work that combines arithmetic circuit complexity, structured matrices and deep learning essentially answers this question. This survey is targeted at complexity theorists who might enjoy reading about how tools developed in arithmetic circuit complexity helped design (to the best of our knowledge) a new family of structured matrices, which in turn seem well-suited for applications in deep learning. However, we hope that folks primarily interested in deep learning would also appreciate the connections to complexity theory.

本调查对算术电路复杂性、结构化矩阵和深度学习的交叉结果进行了必然不完整（且有偏见）的概述。最近有一些研究活动用结构化权重矩阵替换神经网络中的非结构化权重矩阵（目的是减少相应深度学习模型的大小）。这项工作的大部分都是实验性的，在本次调查中，我们将研究问题形式化，并展示了最近一项结合了算术电路复杂性、结构化矩阵和深度学习的工作如何从根本上回答了这个问题。本调查针对的是复杂性理论家，他们可能喜欢阅读有关算术电路复杂性中开发的工具如何帮助设计（据我们所知）新的结构化矩阵系列，这反过来似乎非常适合深度学习中的应用。然而，我们希望主要对深度学习感兴趣的人也能理解与复杂性理论的联系。