Purpose

The expanding capabilities of surgical systems bring with them increasing complexity in the interfaces that humans use to control them. Robotic C-arm X-ray imaging systems, for instance, often require manipulation of independent axes via joysticks, while higher-level control options hide inside device-specific menus. The complexity of these interfaces hinder “ready-to-hand” use of high-level functions. Natural language offers a flexible, familiar interface for surgeons to express their desired outcome rather than remembering the steps necessary to achieve it, enabling direct access to task-aware, patient-specific C-arm functionality.

Methods

We present an English language voice interface for controlling a robotic X-ray imaging system with task-aware functions for pelvic trauma surgery. Our fully integrated system uses a large language model (LLM) to convert natural spoken commands into machine-readable instructions, enabling low-level commands like “Tilt back a bit,” to increase the angular tilt or patient-specific directions like, “Go to the obturator oblique view of the right ramus,” based on automated image analysis.

Results

We evaluate our system with 212 prompts provided by an attending physician, in which the system performed satisfactory actions 97% of the time. To test the fully integrated system, we conduct a real-time study in which an attending physician placed orthopedic hardware along desired trajectories through an anthropomorphic phantom, interacting solely with an X-ray system via voice.

Conclusion

Voice interfaces offer a convenient, flexible way for surgeons to manipulate C-arms based on desired outcomes rather than device-specific processes. As LLMs grow increasingly capable, so too will their applications in supporting higher-level interactions with surgical assistance systems.

中文翻译：

---

开枪吧！手术中智能机器人 X 射线系统的自然语言控制

目的

手术系统功能的不断扩展使得人类用于控制它们的界面变得越来越复杂。例如，机器人 C 形臂 X 射线成像系统通常需要通过操纵杆操纵独立轴，而更高级别的控制选项隐藏在设备特定的菜单中。这些接口的复杂性阻碍了高级功能的“即用”使用。自然语言为外科医生提供了一个灵活、熟悉的界面来表达他们想要的结果，而不是记住实现它所需的步骤，从而能够直接访问任务感知、患者特定的 C 臂功能。

方法

我们提出了一种英语语音界面，用于控制具有任务感知功能的机器人 X 射线成像系统，用于骨盆创伤手术。我们完全集成的系统使用大型语言模型 (LLM) 将自然口头命令转换为机器可读指令，从而启用“向后倾斜一点”等低级命令，以增加倾斜角度或针对患者的特定方向，如“走”右升支的闭孔斜视图”，基于自动图像分析。

结果

我们使用主治医生提供的 212 条提示来评估我们的系统，其中系统在 97% 的时间内执行了令人满意的操作。为了测试完全集成的系统，我们进行了一项实时研究，其中主治医生通过拟人模型将矫形硬件沿着所需的轨迹放置，仅通过语音与 X 射线系统交互。

结论

语音界面为外科医生提供了一种方便、灵活的方式来根据期望的结果而不是特定于设备的过程来操纵 C 形臂。随着法学硕士的能力越来越强，他们在支持与手术辅助系统的更高级别交互方面的应用也将越来越强。