INTRODUCTION Funders must make difficult decisions about which squared treatments to prioritize for randomized trials. Earlier research suggests that experts have no ability to predict which treatments will vindicate their promise. We tested whether a brief training module could improve experts' trial predictions. METHODS We randomized a sample of breast cancer and hematology-oncology experts to the presence or absence of a feedback training module where experts predicted outcomes for five recently completed randomized controlled trials and received feedback on accuracy. Experts then predicted primary outcome attainment for a sample of ongoing randomized controlled trials. Prediction skill was assessed by Brier scores, which measure the average deviation between their predictions and actual outcomes. Secondary outcomes were discrimination (ability to distinguish between positive and non-positive trials) and calibration (higher predictions reflecting higher probability of trials being positive). RESULTS A total of 148 experts (46 for breast cancer, 54 for leukemia, and 48 for lymphoma) were randomized between May and December 2017 and included in the analysis (1217 forecasts for 25 trials). Feedback did not improve prediction skill (mean Brier score for control: 0.22, 95% confidence interval = 0.20-0.24 vs feedback arm: 0.21, 95% confidence interval = 0.20-0.23; p = 0.51). Control and feedback arms showed similar discrimination (area under the curve = 0.70 vs 0.73, p = 0.24) and calibration (calibration index = 0.01 vs 0.01, p = 0.81). However, experts in both arms offered predictions that were significantly more accurate than uninformative forecasts of 50% (Brier score = 0.25). DISCUSSION A short training module did not improve predictions for cancer trial results. However, expert communities showed unexpected ability to anticipate positive trials.Pre-registration record: https://aspredicted.org/4ka6r.pdf.

中文翻译：

---

反馈训练对癌症临床试验结果预测的影响。

简介 资助者必须就随机试验优先考虑哪些平方治疗做出艰难的决定。早期的研究表明，专家没有能力预测哪些治疗方法能够证明他们的承诺。我们测试了简短的培训模块是否可以改善专家的试验预测。方法 我们将乳腺癌和血液肿瘤学专家的样本随机分配到是否存在反馈培训模块，在该模块中，专家预测了最近完成的五项随机对照试验的结果，并收到了准确性反馈。然后，专家预测了正在进行的随机对照试验样本的主要结果达到情况。预测技能通过 Brier 分数进行评估，该分数衡量预测与实际结果之间的平均偏差。次要结果是区分（区分阳性和非阳性试验的能力）和校准（较高的预测反映试验呈阳性的概率较高）。结果 2017 年 5 月至 12 月期间，共有 148 名专家（46 名乳腺癌专家、54 名白血病专家和 48 名淋巴瘤专家）被随机分组​​并纳入分析（25 项试验的 1217 项预测）。反馈并没有提高预测技能（控制的平均 Brier 分数：0.22，95% 置信区间 = 0.20-0.24 对比反馈组：0.21，95% 置信区间 = 0.20-0.23；p = 0.51）。控制臂和反馈臂显示出相似的辨别力（曲线下面积 = 0.70 vs 0.73，p = 0.24）和校准（校准指数 = 0.01 vs 0.01，p = 0.81）。然而，两组专家提供的预测都比 50% 的无信息预测准确得多（Brier 评分 = 0.25）。讨论 简短的训练模块并没有改善对癌症试验结果的预测。然而，专家社区表现出了意想不到的预测积极试验的能力。预注册记录：https://aspredicted.org/4ka6r.pdf。