Abstract

While code review is a critical component of modern software quality assurance, defects can still slip through the review process undetected. Previous research suggests that the main reason for this is a lack of reviewer awareness about the likelihood of defects in proposed changes; even experienced developers may struggle to evaluate the potential risks. If a change’s riskiness is underestimated, it may not receive adequate attention during review, potentially leading to defects being introduced into the codebase. In this paper, we investigate how risk assessment analytics can influence the level of awareness among developers regarding the potential risks associated with code changes; we also study how effective and efficient reviewers are at detecting defects during code review with the use of such analytics. We conduct a controlled experiment using Gherald, a risk assessment prototype tool that analyzes the riskiness of change sets based on historical data. Following a between-subjects experimental design, we assign participants to the treatment (i.e., with access to Gherald) or control group. All participants are asked to perform risk assessment and code review tasks. Through our experiment with 48 participants, we find that the use of Gherald is associated with statistically significant improvements (one-tailed, unpaired Mann-Whitney U test, \(\alpha \) = 0.05) in developer awareness of riskiness of code changes and code review effectiveness. Moreover, participants in the treatment group tend to identify the known defects more quickly than those in the control group; however, the difference between the two groups is not statistically significant. Our results lead us to conclude that the adoption of a risk assessment tool has a positive impact on code review practices, which provides valuable insights for practitioners seeking to enhance their code review process and highlights the importance for further research to explore more effective and practical risk assessment approaches.

中文翻译：

---

研究风险评估分析对风险意识和代码审查绩效的影响

摘要

虽然代码审查是现代软件质量保证的关键组成部分，但缺陷仍然可能在审查过程中未被发现。先前的研究表明，造成这种情况的主要原因是审阅者缺乏对拟议变更中存在缺陷的可能性的认识；即使是经验丰富的开发人员也可能难以评估潜在风险。如果低估了变更的风险，则在审查期间可能不会得到足够的关注，从而可能导致代码库中引入缺陷。在本文中，我们研究了风险评估分析如何影响开发人员对与代码更改相关的潜在风险的认识水平；我们还研究了审查者在代码审查期间如何使用此类分析有效和高效地检测缺陷。我们使用Gherald进行了一项对照实验，Gherald 是一种风险评估原型工具，可根据历史数据分析变更集的风险性。按照受试者间实验设计，我们将参与者分配到治疗组（即可以接触Gherald）或对照组。所有参与者都被要求执行风险评估和代码审查任务。通过我们对 48 名参与者进行的实验，我们发现Gherald的使用与开发人员对代码更改风险的意识以及统计 上的显着改善相关（单尾、不成对的 Mann-Whitney U 检验，\(\alpha \) = 0.05）。代码审查的有效性。此外，治疗组的参与者往往比对照组的参与者更快地识别已知的缺陷；然而，两组之间的差异并不具有统计学意义。我们的结果使我们得出结论，采用风险评估工具对代码审查实践产生积极影响，这为寻求增强代码审查流程的从业者提供了宝贵的见解，并强调了进一步研究探索更有效和实用风险的重要性评估方法。