Introduction Pneumonia is caused by microbes that establish an infectious process in the lungs. The gold standard for pneumonia diagnosis is radiologist-documented pneumonia-related features in radiology notes that are captured in electronic health records in an unstructured format.

Objective The study objective was to develop a methodological approach for assessing validity of a pneumonia diagnosis based on identifying presence or absence of key radiographic features in radiology reports with subsequent rendering of diagnostic decisions into a structured format.

Methods A pneumonia-specific natural language processing (NLP) pipeline was strategically developed applying Clinical Text Analysis and Knowledge Extraction System (cTAKES) to validate pneumonia diagnoses following development of a pneumonia feature–specific lexicon. Radiographic reports of study-eligible subjects identified by International Classification of Diseases (ICD) codes were parsed through the NLP pipeline. Classification rules were developed to assign each pneumonia episode into one of three categories: “positive,” “negative,” or “not classified: requires manual review” based on tagged concepts that support or refute diagnostic codes.

Results A total of 91,998 pneumonia episodes diagnosed in 65,904 patients were retrieved retrospectively. Approximately 89% (81,707/91,998) of the total pneumonia episodes were documented by 225,893 chest X-ray reports. NLP classified and validated 33% (26,800/81,707) of pneumonia episodes classified as “Pneumonia-positive,” 19% as (15401/81,707) as “Pneumonia-negative,” and 48% (39,209/81,707) as “episode classification pending further manual review.” NLP pipeline performance metrics included accuracy (76.3%), sensitivity (88%), and specificity (75%).

Conclusion The pneumonia-specific NLP pipeline exhibited good performance comparable to other pneumonia-specific NLP systems developed to date.

简介 肺炎是由在肺部建立感染过程的微生物引起的。肺炎诊断的黄金标准是放射科医师记录的放射学笔记中与肺炎相关的特征，这些特征以非结构化格式在电子健康记录中捕获。

目的 研究目的是开发一种评估肺炎诊断有效性的方法，该方法基于识别放射学报告中是否存在关键放射学特征，随后将诊断决定呈现为结构化格式。

方法 采用临床文本分析和知识提取系统 (cTAKES) 战略性地开发了肺炎特异性自然语言处理 (NLP) 管道，以在开发肺炎特征特异性词典后验证肺炎诊断。通过 NLP 管道解析由国际疾病分类 (ICD) 代码确定的符合研究条件的受试者的射线照相报告。制定了分类规则，以根据支持或反驳诊断代码的标记概念将每个肺炎事件分为三个类别之一：“阳性”、“阴性”或“未分类：需要人工审查”。

结果 回顾性检索了 65904 例患者中诊断出的 91998 起肺炎事件。225,893 份胸部 X 光报告记录了大约 89% (81,707/91,998) 的肺炎总发作。NLP 将 33% (26,800/81,707) 的肺炎事件分类并验证为“肺炎阳性”，19% (15401/81,707) 为“肺炎阴性”，48% (39,209/81,707) 为“待定的事件分类”进一步的人工审查。” NLP 管道性能指标包括准确性 (76.3%)、敏感性 (88%) 和特异性 (75%)。

结论 肺炎特异性 NLP 管道表现出与迄今为止开发的其他肺炎特异性 NLP 系统相媲美的良好性能。