The abstraction of a shared memory space over separate CPU and GPU memory domains has eased the burden of portability for many HPC codebases. However, users pay for ease of use provided by system-managed memory with a moderate-to-high performance overhead. NVIDIA Unified Virtual Memory (UVM) is currently the primary real-world implementation of such abstraction and offers a functionally equivalent testbed for in-depth performance study for both UVM and future Linux Heterogeneous Memory Management (HMM) compatible systems. The continued advocacy for UVM and HMM motivates improvement of the underlying system. We focus on UVM-based systems and investigate the root causes of UVM overhead, a non-trivial task due to complex interactions of multiple hardware and software constituents and the desired cost granularity.

In our prior work, we delved deeply into UVM system architecture and showed internal behaviors of page fault servicing in batches. We provided quantitative evaluation of batch handling for various applications under different scenarios, including prefetching and oversubscription. We revealed that the driver workload depends on the interactions among application access patterns, GPU hardware constraints, and host OS components. Host OS components have significant overhead present across implementations, warranting close attention.

This extension furthers our prior study in three aspects: fine-grain cost analysis and breakdown, extension to multiple GPUs, and investigation of platforms with different GPU-GPU interconnects. We take a top-down approach to quantitative batch analysis and uncover how constituent component costs accumulate and overlap, governed by synchronous and asynchronous operations. Our multi-GPU analysis shows reduced cost of GPU-GPU batch workloads compared to CPU-GPU workloads. We further demonstrate that while specialized interconnects, NVLink, can improve batch cost, their benefits are limited by host OS software overhead and GPU oversubscription. This study serves as a proxy for future shared memory systems, such as those that interface with HMM, and the development of interconnects.

在独立的 CPU 和 GPU 内存域上抽象共享内存空间减轻了许多 HPC 代码库的可移植性负担。然而，用户为系统管理内存提供的易用性付出了中等到高性能开销的代价。NVIDIA 统一虚拟内存 (UVM) 目前是此类抽象的主要现实实现，并为 UVM 和未来 Linux 异构内存管理 (HMM) 兼容系统的深入性能研究提供了功能等效的测试平台。UVM 和 HMM 的持续倡导推动了底层系统的改进。我们专注于基于 UVM 的系统，并调查 UVM 开销的根本原因，由于多个硬件和软件成分的复杂交互以及所需的成本粒度，这是一项艰巨的任务。

在我们之前的工作中，我们深入研究了 UVM 系统架构，并批量展示了页面错误服务的内部行为。我们对不同场景下的各种应用程序的批量处理进行了定量评估，包括预取和超额订阅。我们发现驱动程序工作负载取决于应用程序访问模式、GPU 硬件限制和主机操作系统组件之间的交互。主机操作系统组件在实施过程中存在大量开销，值得密切关注。

这一扩展在三个方面进一步深化了我们之前的研究：细粒度成本分析和细分、扩展到多个 GPU 以及对具有不同 GPU-GPU 互连的平台的研究。我们采用自上而下的方法进行定量批量分析，并揭示组成部分成本如何在同步和异步操作的控制下累积和重叠。我们的多 GPU 分析显示，与 CPU-GPU 工作负载相比，GPU-GPU 批量工作负载的成本更低。我们进一步证明，虽然专用互连 NVLink 可以提高批量成本，但其优势受到主机操作系统软件开销和 GPU 超额订阅的限制。这项研究可以作为未来共享内存系统的代理，例如那些与 HMM 接口的系统以及互连的开发。