Background It would be useful to be able to assess the utility of predictive models of continuous values before clinical trials are performed. Objective The aim of the study is to compare metrics to assess the potential clinical utility of models that produce continuous value forecasts. Methods We ran a set of data assimilation forecast algorithms on time series of glucose measurements from neurological intensive care unit patients. We evaluated the forecasts using four sets of metrics: glucose root mean square (RMS) error, a set of metrics on a transformed glucose value, the estimated effect on clinical care based on an insulin guideline, and a glucose measurement error grid (Parkes grid). We assessed correlation among the metrics and created a set of factor models. Results The metrics generally correlated with each other, but those that estimated the effect on clinical care correlated with others the least and were generally associated with their own independent factors. The other metrics appeared to separate into those that emphasized errors in low glucose versus errors in high glucose. The Parkes grid was well correlated with the transformed glucose but not the estimation of clinical care. Discussion Our results indicate that we need to be careful before we assume that commonly used metrics like RMS error in raw glucose or even metrics like the Parkes grid that are designed to measure importance of differences will correlate well with actual effect on clinical care processes. A combination of metrics appeared to explain the most variance between cases. As prediction algorithms move into practice, it will be important to measure actual effects.

中文翻译：

---

评估连续临床值的预测：葡萄糖案例研究

背景 在进行临床试验之前，能够评估连续值预测模型的效用将是有用的。目的 本研究的目的是比较指标，以评估产生连续价值预测的模型的潜在临床效用。方法 我们对神经重症监护病房患者的血糖测量时间序列运行了一组数据同化预测算法。我们使用四组指标评估预测：葡萄糖均方根 (RMS) 误差、一组关于转换后的葡萄糖值的指标、基于胰岛素指南对临床护理的估计效果，以及葡萄糖测量误差网格（Parkes 网格） ）。我们评估了指标之间的相关性并创建了一组因子模型。结果 指标通常相互关联，但那些估计对临床护理的影响的人与其他人的相关性最小，并且通常与他们自己的独立因素相关。其他指标似乎分为强调低血糖错误与高血糖错误的指标。Parkes 网格与转化的葡萄糖有很好的相关性，但与临床护理的估计无关。讨论 我们的结果表明，在我们假设常用的指标（如原始葡萄糖中的 RMS 误差）或什至旨在衡量差异重要性的 Parkes 网格等指标将与临床护理过程的实际效果很好地相关之前，我们需要小心。指标的组合似乎可以解释案例之间的最大差异。随着预测算法进入实践，衡量实际效果将变得很重要。