We propose a systematic analysis of deep neural networks (DNNs) based on a signal processing technique for network parameter removal, in the form of synaptic filters that identifies the fragility, robustness and antifragility characteristics of DNN parameters. Our proposed analysis investigates if the DNN performance is impacted negatively, invariantly, or positively on both clean and adversarially perturbed test datasets when the DNN undergoes synaptic filtering. We define three filtering scores for quantifying the fragility, robustness and antifragility characteristics of DNN parameters based on the performances for (i) clean dataset, (ii) adversarial dataset, and (iii) the difference in performances of clean and adversarial datasets. We validate the proposed systematic analysis on ResNet-18, ResNet-50, SqueezeNet-v1.1 and ShuffleNet V2 x1.0 network architectures for MNIST, CIFAR10 and Tiny ImageNet datasets. The filtering scores, for a given network architecture, identify network parameters that are invariant in characteristics across different datasets over learning epochs. Vice-versa, for a given dataset, the filtering scores identify the parameters that are invariant in characteristics across different network architectures. We show that our synaptic filtering method improves the test accuracy of ResNet and ShuffleNet models on adversarial dataset when only the robust and antifragile parameters are selectively retrained at any given epoch, thus demonstrating applications of the proposed strategy in improving model robustness.

我们提出了基于用于网络参数去除的信号处理技术的深度神经网络（DNN）的系统分析，以突触滤波器的形式识别DNN 参数的脆弱性、鲁棒性和反脆弱性特征。我们提出的分析研究了当 DNN 进行突触过滤时，DNN 性能对干净的和对抗性扰动的测试数据集是否会受到负面、不变或正面的影响。我们根据 (i) 干净数据集、(ii) 对抗数据集和 (iii) 干净数据集和对抗数据集的性能差异，定义了三个过滤分数来量化 DNN 参数的脆弱性、鲁棒性和反脆弱性特征。我们验证了针对 MNIST、CIFAR10 和 Tiny ImageNet 数据集的 ResNet-18、ResNet-50、SqueezeNet-v1.1 和 ShuffleNet V2 x1.0 网络架构所提出的系统分析。对于给定的网络架构，过滤分数可以识别在学习时期的不同数据集上特征不变的网络参数。反之亦然，对于给定的数据集，过滤分数识别在不同网络架构中特征不变的参数。我们表明，当在任何给定时期仅选择性地重新训练鲁棒性和抗脆弱性参数时，我们的突触过滤方法提高了 ResNet 和 ShuffleNet 模型在对抗性数据集上的测试准确性，从而证明了所提出的策略在提高模型鲁棒性方面的应用。