Objective The use of electronic health records (EHRs) for clinical risk prediction is on the rise. However, in many practical settings, the limited availability of task-specific EHR data can restrict the application of standard machine learning pipelines. In this study, we investigate the potential of leveraging language models (LMs) as a means to incorporate supplementary domain knowledge for improving the performance of various EHR-based risk prediction tasks. Methods We propose two novel LM-based methods, namely “LLaMA2-EHR” and “Sent-e-Med.” Our focus is on utilizing the textual descriptions within structured EHRs to make risk predictions about future diagnoses. We conduct a comprehensive comparison with previous approaches across various data types and sizes. Results Experiments across 6 different methods and 3 separate risk prediction tasks reveal that employing LMs to represent structured EHRs, such as diagnostic histories, results in significant performance improvements when evaluated using standard metrics such as area under the receiver operating characteristic (ROC) curve and precision-recall (PR) curve. Additionally, they offer benefits such as few-shot learning, the ability to handle previously unseen medical concepts, and adaptability to various medical vocabularies. However, it is noteworthy that outcomes may exhibit sensitivity to a specific prompt. Conclusion LMs encompass extensive embedded knowledge, making them valuable for the analysis of EHRs in the context of risk prediction. Nevertheless, it is important to exercise caution in their application, as ongoing safety concerns related to LMs persist and require continuous consideration.

中文翻译：

---

使用语言模型进行临床风险预测：好处和注意事项

目的 使用电子健康记录 (EHR) 进行临床风险预测的趋势正在增加。然而，在许多实际环境中，特定任务的 EHR 数据的可用性有限可能会限制标准机器学习流程的应用。在本研究中，我们研究了利用语言模型（LM）作为整合补充领域知识的手段的潜力，以提高各种基于 EHR 的风险预测任务的性能。方法我们提出了两种基于 LM 的新颖方法，即“LLaMA2-EHR”和“Sent-e-Med”。我们的重点是利用结构化电子病历中的文本描述来对未来诊断进行风险预测。我们对各种数据类型和大小的先前方法进行了全面比较。结果跨 6 种不同方法和 3 个独立风险预测任务的实验表明，使用 LM 来表示结构化 EHR（例如诊断历史）在使用接收者操作特征 (ROC) 曲线下面积和精度等标准指标进行评估时会带来显着的性能改进-回忆（PR）曲线。此外，它们还具有一些好处，例如少量学习、处理以前未见过的医学概念的能力以及对各种医学词汇的适应性。然而，值得注意的是，结果可能表现出对特定提示的敏感性。结论 LM 包含广泛的嵌入式知识，使其对于风险预测背景下的 EHR 分析很有价值。然而，在应用中务必谨慎，因为与 LM 相关的持续安全问题仍然存在，需要不断考虑。