In recent years, the short video industry is booming. However, there are still many difficulties in the action generation of virtual characters. We observed that on the short video social platform, “hand gesture dance” is a very popular short video form. However, its development is limited by the professionalism of choreography. In order to solve these problems, we propose an intelligent choreography framework, which can generate new gesture sequences for unseen audio based on pairing data in the database. Our framework adopts multimodal method and obtains excellent results. In additional, we collected and produced the first and largest pair labeled hand gesture dance data set. Various experiments showed that our results not only generate smooth and rich action sequences, but also collect some semantic information contained in the audio.

中文翻译：

---

使用 HGS 生成音乐调节的 2D 手势舞蹈

近年来，短视频行业蓬勃发展。然而，虚拟角色的动作生成仍然存在很多困难。我们观察到，在短视频社交平台上，“手势舞”是一种非常流行的短视频形式。但其发展受到编舞专业性的限制。为了解决这些问题，我们提出了一种智能编排框架，它可以根据数据库中的配对数据为看不见的音频生成新的手势序列。我们的框架采用多模态方法并取得了优异的结果。此外，我们收集并生成了第一个也是最大的一对标记手势舞蹈数据集。各种实验表明，我们的结果不仅生成流畅且丰富的动作序列，而且还收集了音频中包含的一些语义信息。