The continuously growing size of deep neural networks (DNNs) has sparked a surge in research on model compression techniques. Among these techniques, multi-resolution model compression has emerged as a promising approach which can generate multiple DNN models with shared weights and different computational complexity (resolution) through a single training. However, in most existing multi-resolution compression methods, the model structures for different resolutions are either predefined or uniformly controlled. This can lead to performance degradation as they fail to implement systematic compression to achieve the optimal model for each resolution. In this paper, we propose to perform multi-resolution compression from a Bayesian perspective. We design a resolution-aware likelihood and a two-layer prior for the channel masks, which allow joint optimization of the shared weights and the model structure of each resolution. To solve the resulted Bayesian inference problem, we develop a low complexity partial update block variational Bayesian inference (PUB-VBI) algorithm. Furthermore, we extend our proposed method into the arbitrary resolution case by proposing an auxiliary neural network (NN) to learn the mapping from the input resolution to the corresponding channel masks. Simulation results show that our proposed method can outperform the baselines on various NN models and datasets.

中文翻译：

---

深度神经网络的多分辨率模型压缩：变分贝叶斯方法

深度神经网络 (DNN) 规模的不断增长引发了模型压缩技术研究的激增。在这些技术中，多分辨率模型压缩已成为一种有前途的方法，它可以通过单次训练生成具有共享权重和不同计算复杂度（分辨率）的多个 DNN 模型。然而，在大多数现有的多分辨率压缩方法中，不同分辨率的模型结构要么是预定义的，要么是统一控制的。这可能会导致性能下降，因为它们无法实现系统压缩来实现每个分辨率的最佳模型。在本文中，我们建议从贝叶斯角度执行多分辨率压缩。我们为通道掩码设计了分辨率感知的可能性和两层先验，这允许联合优化共享权重和每个分辨率的模型结构。为了解决由此产生的贝叶斯推理问题，我们开发了一种低复杂度的部分更新块变分贝叶斯推理（PUB-VBI）算法。此外，我们通过提出辅助神经网络（NN）来学习从输入分辨率到相应通道掩模的映射，将我们提出的方法扩展到任意分辨率情况。仿真结果表明，我们提出的方法可以优于各种神经网络模型和数据集上的基线。