In video action recognition, effective spatiotemporal modeling is crucial. However, traditional two-stream methods face challenges in integrating spatial information from RGB images and temporary information from optical flow without long-range temporal modelling. To address these limitations, we propose the Deep Fusion Module (DFM), which focuses on the deep fusion of spatial and temporal information and consists of two components. First, we propose an Attention Fusion Module (AFM) to effectively fuse the shallow features obtained from a two-stream network, thereby facilitating the integration of spatial and temporal information. Next, we incorporate a SpatioTemporal Module (STM), comprising a ConvGRU and a 1×1 convolution, to model long-range temporal dependency and fuse spatial-temporal features. Experiments on the UCF101 dataset show that our method achieves 96.5% accuracy, outperforming baseline two-stream models by 0.3%.

在视频动作识别中，有效的时空建模至关重要。然而，传统的双流方法在集成 RGB 图像的空间信息和光流的临时信息而无需进行长程时间建模方面面临着挑战。为了解决这些限制，我们提出了深度融合模块（DFM），它专注于空间和时间信息的深度融合，由两个组件组成。首先，我们提出了一种注意力融合模块（AFM）来有效融合从双流网络获得的浅层特征，从而促进空间和时间信息的整合。接下来，我们结合了一个时空模块（STM），包括一个 ConvGRU 和一个 1×1 卷积，来模拟长程时间依赖性并融合时空特征。在 UCF101 数据集上的实验表明，我们的方法达到了 96.5% 的准确率，比基线双流模型高了 0.3%。