In real-life scenarios, Human Activity Recognition (HAR) from video data is prone to occlusion of one or more body parts of the human subjects involved. Although it is common sense that the recognition of the majority of activities strongly depends on the motion of some body parts, which when occluded compromise the performance of recognition approaches, this problem is often underestimated in contemporary research works. Currently, training and evaluation is based on datasets that have been shot under laboratory (ideal) conditions, i.e. without any kind of occlusion. In this work, we propose an approach for HAR in the presence of partial occlusion, in cases wherein up to two body parts are involved. We assume that human motion is modeled using a set of 3D skeletal joints and also that occluded body parts remain occluded during the whole duration of the activity. We solve this problem using regression, performed by a novel deep Convolutional Recurrent Neural Network (CRNN). Specifically, given a partially occluded skeleton, we attempt to reconstruct the missing information regarding the motion of its occluded part(s). We evaluate our approach using four publicly available human motion datasets. Our experimental results indicate a significant increase of performance, when compared to baseline approaches, wherein networks that have been trained using only nonoccluded or both occluded and nonoccluded samples are evaluated using occluded samples. To the best of our knowledge, this is the first research work that formulates and copes with the problem of HAR under occlusion as a regression task.

在现实生活场景中，视频数据中的人类活动识别 (HAR) 很容易遮挡所涉及的人类受试者的一个或多个身体部位。尽管众所周知，大多数活动的识别强烈依赖于某些身体部位的运动，当这些部位的运动被遮挡时，会损害识别方法的性能，但在当代研究工作中，这个问题常常被低估。目前，训练和评估基于在实验室（理想）条件下拍摄的数据集，即没有任何遮挡。在这项工作中，我们提出了一种在存在部分遮挡（涉及最多两个身体部位的情况下）的 HAR 方法。我们假设人体运动是使用一组 3D 骨骼关节进行建模的，并且遮挡的身体部位在整个活动期间保持遮挡状态。我们使用由新颖的深度卷积循环神经网络（CRNN）执行的回归来解决这个问题。具体来说，给定一个部分遮挡的骨架，我们尝试重建有关其被遮挡部分的运动的丢失信息。我们使用四个公开可用的人体运动数据集来评估我们的方法。我们的实验结果表明，与基线方法相比，性能显着提高，其中仅使用非遮挡样本或使用遮挡样本和非遮挡样本训练的网络使用遮挡样本进行评估。据我们所知，这是第一个将遮挡下的 HAR 问题作为回归任务来制定和处理的研究工作。