In this paper, we propose a new lightweight Channel-Spatial Knowledge Distillation (CSKD) method to handle the task of efficient image semantic segmentation. More precisely, we investigate the KD approach that train a compressed neural network called student under the supervision of a heavy one called teacher. In this context, we propose to improve the distillation mechanism by capturing the contextual dependencies in spatial and channel dimensions through a self-attention principle. In addition, to quantify the difference between the teacher and student knowledge, we adopt the Centered Kernel Alignment (CKA) metric that avoids the student to add additional leaning layers to match the teacher features size. Experimental results over Cityscapes, CamVid and Pascal VOC datasets demonstrate that our method can achieve outstanding performance. The code is available at .

中文翻译：

---

用于高效语义分割的通道空间知识蒸馏

在本文中，我们提出了一种新的轻量级通道空间知识蒸馏（CSKD）方法来处理高效的图像语义分割任务。更准确地说，我们研究了 KD 方法，该方法在称为教师的重磅神经网络的监督下训练称为学生的压缩神经网络。在这种情况下，我们建议通过自注意力原理捕获空间和通道维度中的上下文依赖关系来改进蒸馏机制。此外，为了量化教师和学生知识之间的差异，我们采用中心核对齐（CKA）度量，避免学生添加额外的学习层来匹配教师特征大小。在 Cityscapes、CamVid 和 Pascal VOC 数据集上的实验结果表明，我们的方法可以实现出色的性能。该代码可在 处获取。