ABSTRACT

In recent years, deep learning methods have been widely used for the classification of hyperspectral images. However, their limited availability under the condition of small samples remains a serious issue. Moreover, the current mainstream approaches based on convolutional neural networks do well in local feature extraction but are also restricted by its limited receptive field. Hence, these models are unable to capture long-distance dependencies both on spatial and spectral dimension. To address above issues, this paper proposes a global-local U-Net augmented by transformers (TrmGLU-Net). First, whole hyperspectral images are input to the model for end-to-end training to capture the contextual information. Then, a transformer-augmented U-Net is designed with alternating transformers and convolutional layers to perceive both global and local information. Finally, a superpixel-based label expansion method is proposed to expand the labels and improve the performance under the condition of small samples. Extensive experiments on four hyperspectral scenes demonstrate that TrmGLU-Net has better performance than other advanced patch-level and image-level methods with limited training samples. The relevant code will be opened at https://github.com/sssssyf/TrmGLU-Net

摘要

近年来，深度学习方法被广泛应用于高光谱图像的分类。然而，在小样本情况下，它们的可用性有限仍然是一个严重的问题。此外，目前基于卷积神经网络的主流方法在局部特征提取方面表现良好，但也受到其有限的感受野的限制。因此，这些模型无法捕获对空间和光谱维度的长距离依赖性。为了解决上述问题，本文提出了一种由变压器增强的全局局部 U-Net（TrmGLU-Net）。首先，将整个高光谱图像输入到模型中进行端到端训练以捕获上下文信息。然后，Transformer-augmented U-Net 设计有交替的 Transformer 和卷积层来感知全局和局部信息。最后，提出一种基于超像素的标签扩展方法，以扩展标签，提高小样本条件下的性能。对四个高光谱场景的大量实验表明，TrmGLU-Net 在训练样本有限的情况下比其他先进的块级和图像级方法具有更好的性能。相关代码将在https://github.com/sssssyf/TrmGLU-Net打开 对四个高光谱场景的大量实验表明，TrmGLU-Net 在训练样本有限的情况下比其他先进的块级和图像级方法具有更好的性能。相关代码将在https://github.com/sssssyf/TrmGLU-Net打开 对四个高光谱场景的大量实验表明，TrmGLU-Net 在训练样本有限的情况下比其他先进的块级和图像级方法具有更好的性能。相关代码将在https://github.com/sssssyf/TrmGLU-Net打开