In this position paper, we discuss the potential for leveraging LLMs as interactive research tools to facilitate collaboration between human coders and AI to effectively annotate online risk data at scale. Collaborative human-AI labeling is a promising approach to annotating large-scale and complex data for various tasks. Yet, tools and methods to support effective human-AI collaboration for data annotation are under-studied. This gap is pertinent because co-labeling tasks need to support a two-way interactive discussion that can add nuance and context, particularly in the context of online risk, which is highly subjective and contextualized. Therefore, we provide some of the early benefits and challenges of using LLMs-based tools for risk annotation and suggest future directions for the HCI research community to leverage LLMs as research tools to facilitate human-AI collaboration in contextualized online data annotation. Our research interests align very well with the purposes of the LLMs as Research Tools workshop to identify ongoing applications and challenges of using LLMs to work with data in HCI research. We anticipate learning valuable insights from organizers and participants into how LLMs can help reshape the HCI community's methods for working with data.

中文翻译：

---

利用大型语言模型 (LLM) 支持人类与人工智能协作在线风险数据注释

在这篇立场文件中，我们讨论了利用法学硕士作为交互式研究工具来促进人类编码员和人工智能之间的协作以有效地大规模注释在线风险数据的潜力。人机协作标记是一种很有前途的方法，可以为各种任务注释大规模且复杂的数据。然而，支持有效的人类与人工智能协作进行数据注释的工具和方法尚未得到充分研究。这种差距是相关的，因为联合标记任务需要支持双向互动讨论，可以增加细微差别和背景，特别是在高度主观和情境化的在线风险背景下。因此，我们提供了使用基于法学硕士的工具进行风险注​​释的一些早期好处和挑战，并为人机交互研究社区提出了未来的方向，以利用法学硕士作为研究工具，促进情境化在线数据注释中的人类与人工智能的协作。我们的研究兴趣与法学硕士作为研究工具研讨会的目的非常吻合，旨在确定使用法学硕士在人机交互研究中处理数据的持续应用和挑战。我们期望从组织者和参与者那里学习法学硕士如何帮助重塑 HCI 社区处理数据的方法的宝贵见解。