Background

Objectives of this study were to analyze the impact of including librarians and information specialist as methodological peer-reviewers. We sought to determine if and how librarians’ comments differed from subject peer-reviewers’; whether there were differences in the implementation of their recommendations; how this impacted editorial decision-making; and the perceived utility of librarian peer-review by librarians and authors.

Methods

We used a mixed method approach, conducting a qualitative analysis of reviewer reports, author replies and editors’ decisions of submissions to the International Journal of Health Governance. Our content analysis categorized 16 thematic areas, so that methodological and subject peer-reviewers’ comments, decisions and rejection rates could be compared. Categories were based on the standard areas covered in peer-review (e.g., title, originality, etc.) as well as additional in-depth categories relating to the methodology (e.g., search strategy, reporting guidelines, etc.). We developed and used criteria to judge reviewers’ perspectives and code their comments.

We conducted two online multiple-choice surveys which were qualitatively analyzed: one of methodological peer-reviewers’ perceptions of peer-reviewing, the other of published authors’ views on the suggested revisions.

Results

Methodological peer-reviewers assessed 13 literature reviews submitted between September 2020 and March 2023. 55 reviewer reports were collected: 25 from methodological peer-reviewers, 30 from subject peer-reviewers (mean: 4.2 reviews per manuscript). Methodological peer-reviewers made more comments on methodologies, with authors more likely to implement their changes (52 of 65 changes, vs. 51 of 82 by subject peer-reviewers); they were also more likely to reject submissions (seven vs. four times, respectively). Where there were differences in recommendations to editors, journal editors were more likely to follow methodological peer-reviewers (nine vs. three times, respectively). The survey of published authors (87.5% response rate) revealed four of seven found comments on methodologies helpful. Librarians’ survey responses (66.5% response rate) revealed those who conducted peer-reviews felt they improved quality of publications.

Conclusions

Librarians can enhance evidence synthesis publications by ensuring methodologies have been conducted and reported appropriately. Their recommendations helped authors revise submissions and facilitated editorial decision-making. Further research could determine if sharing reviews with subject peer-reviewers and journal editors could benefit them in better understanding of evidence synthesis methodologies.

中文翻译：

---

图书馆员和信息专家作为方法同行评审员：《国际健康治理杂志》的案例研究

背景

本研究的目的是分析图书馆员和信息专家作为方法同行评审员的影响。我们试图确定图书馆员的评论与主题同行评审员的评论是否以及如何不同；他们的建议的实施是否存在差异；这如何影响编辑决策；以及图书馆员和作者对图书馆员同行评审的认知效用。

方法

我们采用混合方法，对审稿人报告、作者回复和编辑向《国际健康治理杂志》提交的决定进行定性分析。我们的内容分析分为 16 个主题领域，以便可以比较方法论和主题同行评审员的评论、决定和拒绝率。类别基于同行评审涵盖的标准领域（例如，标题、原创性等）以及与方法相关的其他深入类别（例如，检索策略、报告指南等）。我们制定并使用标准来判断评审者的观点并对他们的评论进行编码。

我们进行了两项在线多项选择调查，并进行了定性分析：一项是方法论同行评审员对同行评审的看法，另一项是已发表作者对建议修订的看法。

结果

方法学同行评审员评估了 2020 年 9 月至 2023 年 3 月期间提交的 13 篇文献综述。收集了 55 份评审员报告：25 份来自方法学同行评审员，30 份来自主题同行评审员（平均：每份稿件 4.2 条评审）。方法论同行评审员对方法论提出了更多评论，作者更有可能实施他们的更改（65 项更改中有 52 项，而主题同行评审员的 82 项更改中有 51 项）；他们也更有可能拒绝提交（分别是七次和四次）。如果对编辑的建议存在差异，期刊编辑更有可能遵循方法论同行评审员的意见（分别为 9 次和 3 次）。对已发表作者的调查（87.5% 的回复率）显示，七分之四的作者认为对方法论的评论有帮助。图书馆员的调查答复（答复率为 66.5%）显示，进行同行评审的图书馆员认为他们提高了出版物的质量。

结论

图书馆员可以通过确保正确实施和报告方法来增强证据综合出版物。他们的建议帮助作者修改投稿并促进编辑决策。进一步的研究可以确定与主题同行评审员和期刊编辑分享评论是否有利于他们更好地理解证据合成方法。