International organizations (IOs) of the United Nations (UN) system publish around 750 evaluation reports per year, offering insights on their performance across project, program, institutional, and thematic activities. So far, it was not feasible to extract quantitative performance measures from these text-based reports. Using deep learning, this article presents a novel text-based performance metric: We classify individual sentences as containing a negative, positive, or neutral assessment of the evaluated IO activity and then compute the share of positive sentences per report. Content validation yields that the measure adequately reflects the underlying concept of performance; convergent validation finds high correlation with human-provided performance scores by the World Bank; and construct validation shows that our measure has theoretically expected results. Based on this, we present a novel dataset with performance measures for 1,082 evaluated activities implemented by nine UN system IOs and discuss avenues for further research.

联合国 (UN) 系统的国际组织 (IO) 每年发布约 750 份评估报告，提供有关其在项目、计划、机构和主题活动中的绩效的见解。到目前为止，从这些基于文本的报告中提取定量绩效指标是不可行的。本文使用深度学习提出了一种新颖的基于文本的性能指标：我们将单个句子分类为包含对评估的 IO 活动的负面、正面或中性评估，然后计算每个报告中正面句子的份额。内容验证表明该措施充分反映了绩效的基本概念；收敛验证发现世界银行与人类提供的绩效分数高度相关；和构造验证表明我们的措施具有理论上预期的结果。在此基础上，我们提出了一个新的数据集，其中包含九个联合国系统 IO 实施的 1,082 项评估活动的绩效衡量标准，并讨论了进一步研究的途径。