Although peer review is a widely-used pedagogical technique, its value depends upon the quality of the reviews that students produce, and much research remains to be done to systematically study the nature, causes, and consequences of variation in peer review quality. We propose a new framework that conceptualizes five larger dimensions of peer review quality and then present a study that investigated three specific peer review quality constructs in a large dataset and further explored how these constructs change through different types of self-regulation peer reviewing experiences. Peer review data across multiple assignments were analyzed from 2,092 undergraduate students enrolled in one of three offerings of a biology course at a large public research university in the United States. Peer review quality was measured in terms of comment amount, comment accuracy, and rating accuracy; the measures of reviewing experience focused upon self-regulated learning factors such as practice, feedback, others’ modeling, and relative performance. Meta-correlation (for testing reliability, separability, and stability) and -regression (as a time-series analysis for testing the relationship of changes across assignments in reviewing quality with experiences as reviewer and reviewee) are used to establish the robustness of effects and meaningful variation of effects across course offerings and assignments. Results showed that there were three meaningful review quality constructs (i.e., were measured reliably, separable, and semi-stable over time). Further, all three showed changes in response to previous reviewer and reviewee experiences, but only feedback helpfulness, in particular, showed effects of all four examined types of self-regluation experiences (practice, feedback, others’ modeling, and relative performance). The findings suggest that instructors can improve review quality by providing comment prompt scaffolds that lead to longer comments as well as by matching authors with similarly performing reviewers.

中文翻译：

---

通过先前同伴反馈任务中不同维度的经验对同伴反馈质量方面进行自我调节

尽管同行评审是一种广泛使用的教学技术，但其价值取决于学生发表的评审的质量，并且仍然需要进行大量研究来系统地研究同行评审质量变化的性质、原因和后果。我们提出了一个新的框架，该框架概念化了同行评审质量的五个更大维度，然后提出了一项研究，调查了大型数据集中的三个特定同行评审质量结构，并进一步探讨了这些结构如何通过不同类型的自我监管同行评审经验而变化。对美国一所大型公立研究型大学生物课程三门课程之一注册的 2,092 名本科生的多项作业的同行评审数据进行了分析。同行评审质量通过评论数量、评论准确性和评级准确性来衡量；回顾经验的措施侧重于自我调节的学习因素，例如实践、反馈、他人的榜样和相对表现。元相关（用于测试可靠性、可分离性和稳定性）和回归（作为一种时间序列分析，用于测试质量评审中任务之间的变化与评审者和被评审者的经验之间的关系）用于建立效果的稳健性和不同课程和作业之间的影响存在有意义的变化。结果表明，存在三种有意义的评审质量结构（即随着时间的推移可靠地测量、可分离和半稳定）。此外，所有三个都显示了对先前审阅者和被审阅者体验的响应的变化，但只有反馈有用性，特别是显示了所有四种检查类型的自我调节体验（实践、反馈、他人的建模和相对表现）的影响。研究结果表明，教师可以通过提供导致更长评论的评论提示支架以及将作者与表现相似的审稿人进行匹配来提高审稿质量。