Abstract

Diffusion models are effective purification methods, where the noises or adversarial attacks are removed using generative approaches before pre-existing classifiers conducting classification tasks. However, the efficiency of diffusion models is still a concern, and existing solutions are based on knowledge distillation which can jeopardize the generation quality because of the small number of generation steps. Hence, we propose TendiffPure as a tensorized and compressed diffusion model for purification. Unlike the knowledge distillation methods, we directly compress U-Nets as backbones of diffusion models using tensor-train decomposition, which reduces the number of parameters and captures more spatial information in multi-dimensional data such as images. The space complexity is reduced from O(N²) to O(NR²) with R ≤ 4 as the tensor-train rank and N as the number of channels. Experimental results show that TendiffPure can more efficiently obtain high-quality purification results and outperforms the baseline purification methods on CIFAR-10, Fashion-MNIST, and MNIST datasets for two noises and one adversarial attack.

中文翻译：

---

TendiffPure：用于净化的卷积张量训练去噪扩散模型

摘要

扩散模型是有效的净化方法，在预先存在的分类器执行分类任务之前，使用生成方法消除噪声或对抗性攻击。然而，扩散模型的效率仍然是一个问题，现有的解决方案基于知识蒸馏，由于生成步骤数量较少，这可能会危及生成质量。因此，我们提出 TendiffPure 作为用于净化的张量化和压缩扩散模型。与知识蒸馏方法不同，我们使用张量训练分解直接压缩 U-Net 作为扩散模型的主干，这减少了参数数量并捕获图像等多维数据中的更多空间信息。空间复杂度从O ( N ² )降低到O ( NR ² )，其中R ≤ 4 作为张量序列秩，N作为通道数。实验结果表明，对于两种噪声和一种对抗性攻击，TendiffPure 可以更有效地获得高质量的纯化结果，并且在 CIFAR-10、Fashion-MNIST 和 MNIST 数据集上优于基线纯化方法。