We present the new software OpDiLib, a universal add-on for classical operator overloading AD tools that enables the automatic differentiation (AD) of OpenMP parallelized code. With it, we establish support for OpenMP features in a reverse mode operator overloading AD tool to an extent that was previously only reported on in source transformation tools. We achieve this with an event-based implementation ansatz that is unprecedented in AD. Combined with modern OpenMP features around OMPT, we demonstrate how it can be used to achieve differentiation without any additional modifications of the source code; neither do we impose a priori restrictions on the data access patterns, which makes OpDiLib highly applicable. For further performance optimizations, restrictions like atomic updates on adjoint variables can be lifted in a fine-grained manner. OpDiLib can also be applied in a semi-automatic fashion via a macro interface, which supports compilers that do not implement OMPT. We demonstrate the applicability of OpDiLib for a pure operator overloading approach in a hybrid parallel environment. We quantify the cost of atomic updates on adjoint variables and showcase the speedup and scaling that can be achieved with the different configurations of OpDiLib in both the forward and the reverse pass.

我们展示了新软件 OpDiLib，它是经典运算符重载 AD 工具的通用插件，可实现 OpenMP 并行代码的自动微分 (AD)。有了它，我们在反向模式运算符重载 AD 工具中建立了对 OpenMP 功能的支持，达到了以前仅在源转换工具中报告过的程度。我们通过基于事件的实施 ansatz 来实现这一点，这在 AD 中是前所未有的。结合围绕 OMPT 的现代 OpenMP 功能，我们演示了如何在不对源代码进行任何额外修改的情况下使用它来实现差异化；我们也没有对数据访问模式施加先验限制，这使得 OpDiLib 非常适用。为了进一步优化性能，可以以细粒度的方式取消对伴随变量的原子更新等限制。OpDiLib 还可以通过宏接口以半自动方式应用，该接口支持不实现 OMPT 的编译器。我们展示了 OpDiLib 在混合并行环境中对纯运算符重载方法的适用性。我们量化了伴随变量的原子更新成本，并展示了在正向和反向传递中使用 OpDiLib 的不同配置可以实现的加速和缩放。