Security vulnerability fixes could be a promising research avenue for Automated Program Repair (APR) techniques. In recent years, APR tools have been thoroughly developed for fixing generic bugs. However, the area is still relatively unexplored when it comes to fixing security bugs or vulnerabilities. In this paper, we evaluate nine state-of-the-art APR tools and one vulnerability-specific repair tool. In particular, we investigate their ability to generate patches for 79 real-world Java vulnerabilities in the Vul4J dataset, as well as the level of trustworthiness of these patches. We evaluate the tools with respect to their ability to generate security patches that are (i) testable, (ii) having the positive effect of closing the vulnerability, and (iii) not having side effects from a functional point of view. Our results show that the evaluated APR tools were able to generate testable patches for around 20% of the considered vulnerabilities. On average, nearly 73% of the testable patches indeed eliminate the vulnerabilities, but only 44% of them could actually fix security bugs while maintaining the functionalities. To understand the root cause of this phenomenon, we conduct a detailed comparative study of the general bug fix patterns in Defect4J and the vulnerability fix patterns in ExtraVul (which we extend from Vul4J). Our investigation shows that, although security patches are short in terms of lines of code, they contain unique characteristics in their fix patterns compared to general bugs. For example, many security fixes require adding method calls. These method calls contain specific input validation-related keywords, such as encode, normalize, and trim. In this regard, our study suggests that additional repair patterns should be implemented for existing APR tools to fix more types of security vulnerabilities.

安全漏洞修复可能是自动程序修复（APR）技术的一个有前途的研究途径。近年来，APR 工具已得到彻底开发，用于修复一般错误。然而，在修复安全错误或漏洞方面，该领域仍然相对未经探索。在本文中，我们评估了九种最先进的 APR 工具和一种针对漏洞的修复工具。我们特别调查了他们为 Vul4J 数据集中的 79 个真实 Java 漏洞生成补丁的能力，以及这些补丁的可信度。我们评估这些工具生成安全补丁的能力，这些补丁是（i）可测试的，（ii）具有关闭漏洞的积极作用，以及（iii）从功能角度来看没有副作用。我们的结果表明，经过评估的 APR 工具能够为大约 20% 的所考虑的漏洞生成可测试的补丁。平均而言，近 73% 的可测试补丁确实消除了漏洞，但只有 44% 的补丁能够在保持功能的同时真正修复安全错误。为了了解这种现象的根本原因，我们对 Defect4J 中的一般错误修复模式和 ExtraVul（我们从 Vul4J 扩展而来）中的漏洞修复模式进行了详细的比较研究。我们的调查表明，尽管安全补丁的代码行数很短，但与一般错误相比，它们的修复模式包含独特的特征。例如，许多安全修复需要添加方法调用。这些方法调用包含特定的输入验证相关关键字，例如encode、normalize和trim。对此，我们的研究建议，应该对现有的 APR 工具实施额外的修复模式，以修复更多类型的安全漏洞。