This paper introduces Jubileo, an open-source simulated humanoid robot as a framework for the development of human-robot interaction applications. By leveraging the power of the Robot Operating System (ROS) and Unity in a virtual reality environment, this simulation establishes a strong connection to real robotics, faithfully replicating the robot’s physical components down to its motors and enabling communication with servo-actuators to control both the animatronic face and the joints of a real humanoid robot. To validate the capabilities of the framework, we propose English teaching games that integrate Virtual Reality (VR), game-based Human-Robot Interaction (HRI), and advanced large language models such as Generative Pre-trained Transformer (GPT). These games aim to foster linguistic competence within dynamic and interactive virtual environments. The incorporation of large language models bolsters the robot’s capability to generate human-like responses, thus facilitating a more realistic conversational experience. Moreover, the simulation framework reduces real-world testing risks and offers a cost-effective, efficient, and scalable platform for developing new HRI applications. The paper underscores the transformative potential of converging VR, large language models, and HRI, particularly in educational applications.

本文介绍了 Jubileo，一个开源模拟人形机器人，作为开发人机交互应用程序的框架。通过在虚拟现实环境中利用机器人操作系统 (ROS) 和 Unity 的强大功能，该模拟与真实机器人技术建立了牢固的联系，忠实地复制了机器人的物理组件直至其电机，并实现与伺服执行器的通信以控制两者真实人形机器人的电子动画脸和关节。为了验证该框架的功能，我们提出了集成虚拟现实（VR）、基于游戏的人机交互（HRI）和高级大语言模型（例如生成预训练变压器（GPT））的英语教学游戏。这些游戏旨在培养动态和交互式虚拟环境中的语言能力。大型语言模型的结合增强了机器人生成类似人类响应的能力，从而促进更真实的对话体验。此外，模拟框架降低了现实世界的测试风险，并为开发新的 HRI 应用程序提供了一个经济高效、高效且可扩展的平台。该论文强调了 VR、大型语言模型和 HRI 融合的变革潜力，特别是在教育应用方面。