Background

Peer review is essential to the advancement of knowledge. However, training on how to conduct peer review is limited, unorganized, and not well studied. Thus, we sought to determine if a structured mentored peer-review program improved peer review training as measured by multiple quantitative and qualitative assessments.

Methods

This pre-post intervention study enrolled 55 mentees across 5 cohorts from 2020 to 2023. Each cohort completed pre-program evaluations, participated in 2 mentored reviews, and completed post-program evaluations over 6 months. Mentors and mentees completed pre-program demographic and review experience questionnaires. Outcome measures included (1) total and sub-scores on the modified Review Quality Index (mRQI) applied to the same pre-selected research manuscript reviewed by mentees both pre and post intervention, (2) mentee self-perceived comfort with and understanding of the review process using a custom questionnaire, and (3) mentor satisfaction surveys. Pre- and post-program measures were compared using the Wilcoxon signed-rank test.

Results

Post-program total modified RQI score (median (IQR) = 31 (26.3–35.8)) was higher than pre-program total score (26.6 (19.7–29.7)) for the 42 mentees who completed both pre- and post-program reviews. Mentees reported improved perception of review (median (IQR) pre = 4 (3–4), post = 5 (4–5)) and editorial processes (pre = 3 (2–4), post = 4 (4–5)) as well as self-perceived confidence in completing an independent review of both scientific (median (IQR) pre = 2 (2–3), post = 4 (4–4)) and non-scientific (pre = 3 (2–4), post = 4 (4–5)) manuscripts following program participation. p < 0.0001 for all scores noted. Mentors reported high scores for enjoyment (median (range) 5/5 (3–5)) and interest in repeat participation (5/5 (2–5)).

Conclusions

A 6-month structured mentored-review program including 2 mentored reviews improves peer review training as measured by the modified RQI as well as participant self-perceived understanding of publication science with high mentor satisfaction.

中文翻译：

---

结构化的、以期刊为主导的同行评审指导计划增强了同行评审培训

背景

同行评审对于知识的进步至关重要。然而，关于如何进行同行评审的培训有限、无组织且没有得到充分研究。因此，我们试图通过多种定量和定性评估来确定结构化指导同行评审计划是否改善了同行评审培训。

方法

这项前后干预研究从 2020 年到 2023 年招募了 5 个队列的 55 名受训者。每个队列都完成了计划前评估，参加了 2 次指导性审查，并在 6 个月内完成了计划后评估。导师和学员完成了计划前的人口统计和回顾经验调查问卷。结果衡量标准包括 (1) 修改后的审阅质量指数 (mRQI) 的总分和分项分数，该指数适用于受指导者在干预前和干预后审阅的同一预选研究手稿，(2) 受指导者自我感知的舒适度和理解使用定制调查问卷的审查过程，以及 (3) 导师满意度调查。使用 Wilcoxon 符号秩检验对计划前和计划后的测量进行比较。

结果

完成项目前和项目后审核的 42 名受训者的项目后修正 RQI 总分（中位数 (IQR) = 31 (26.3–35.8)）高于项目前总分 (26.6 (19.7–29.7)) 。受指导者报告对审稿（中位数 (IQR) 前 = 4 (3–4)，后 = 5 (4–5)）和编辑流程（前 = 3 (2–4)，后 = 4 (4–5)）的看法有所改善）以及完成对科学（中位数（IQR）前 = 2 (2–3)，后 = 4 (4–4)）和非科学（前 = 3 (2– 4), 参与项目后发表 = 4 (4–5)) 份手稿。 对于所有记录的分数，p < 0.0001。导师在享受方面得分很高（中位数（范围）5/5 (3-5)），在重复参与的兴趣方面得分很高（5/5 (2-5)）。

结论

为期 6 个月的结构化指导评审计划包括 2 次指导评审，通过修改后的 RQI 以及参与者对出版科学的自我认知理解来衡量，改善了同行评审培训，导师满意度很高。