Abstract

Human behavior recognition is a hot research topic in the field of computer vision, and a complete behavior recognition usually includes human detection, human tracking and behavior recognition. At present, the two tasks of human tracking and abnormal behavior recognition based on deep learning are mostly executed separately, and the related feature information in the two tasks cannot be fully utilized, resulting in high time cost and resource consumption of the final abnormal behavior recognition algorithm. The problem greatly limits the widespread application of abnormal behavior recognition. In order to improve the performance of the algorithm a novel model for abnormal behaviors recognition based on human target tracking is proposed, which implements the process of recognizing abnormal behaviors after human target tracking through feature sharing. First, the real-time multi-domain convolutional neural network is improved by introducing a spatial attention mechanism to improve its tracking of a particular human body in a video series. Then the output of the convolutional layer in MDnet is used as the input of the abnormal behavior recognition network, and these features are combined with CNN and LSTM to realize human abnormal behavior recognition. During the network training process, a multi-task learning approach was used to train a model for human tracking and behaviour recognition. Six types of abnormal behaviors selected on the CASIA Behavioural Analytics dataset and 12 types of behaviours selected on the NTU database are used to train and test the network model. According to test results, the proposed model is capable of tracking human targets precisely and in real time (26 frames per second). The proposed model can also distinguish abnormal behaviors of tracking targets with a recognition rate of 92.1%. The human features obtained in the tracking model is used as the input of the abnormal behavior recognition network, so the feature sharing of tracking and recognition is achieved, and a complete abnormal behavior recognition framework including target tracking, feature extraction, and behavior recognition is established. There is great practical significance to the proposed method.

中文翻译：

---

一种基于共享人体目标跟踪特征的异常行为识别算法

摘要

人体行为识别是计算机视觉领域的研究热点，一个完整的行为识别通常包括人体检测、人体跟踪和行为识别。目前基于深度学习的人体跟踪和异常行为识别这两个任务大多是分开执行的，两个任务中的相关特征信息无法得到充分利用，导致最终异常行为识别的时间成本和资源消耗较高算法。该问题极大地限制了异常行为识别的广泛应用。为了提高算法的性能，提出了一种基于人体目标跟踪的异常行为识别模型，通过特征共享实现了人体目标跟踪后的异常行为识别过程。首先，通过引入空间注意机制来改进实时多域卷积神经网络，以改善其对视频系列中特定人体的跟踪。然后将MDnet中卷积层的输出作为异常行为识别网络的输入，将这些特征与CNN和LSTM相结合，实现人体异常行为识别。在网络训练过程中，采用多任务学习方法来训练人体跟踪和行为识别模型。使用CASIA行为分析数据集上选取的6种异常行为和NTU数据库上选取的12种行为来训练和测试网络模型。根据测试结果，该模型能够精确、实时（每秒26帧）跟踪人体目标。该模型还可以区分跟踪目标的异常行为，识别率为92.1%。将跟踪模型中获得的人体特征作为异常行为识别网络的输入，实现跟踪与识别的特征共享，建立了包括目标跟踪、特征提取、行为识别的完整的异常行为识别框架。该方法具有重要的实际意义。