Graphics Processing Units (GPUs) are the accelerator of choice in a variety of application domains because they can accelerate massively parallel workloads and can be easily programmed using general-purpose programming frameworks such as CUDA and OpenCL. Each Streaming Multiprocessor (SM) contains an L1 data cache (L1D) to exploit the locality in data accesses. L1D misses are costly for GPUs due to two reasons. First, L1D misses consume a lot of energy as they need to access the L2 cache (L2) via an on-chip network and the off-chip DRAM in case of L2 misses. Second, L1D misses impose performance overhead if the GPU does not have enough active warps to hide the long memory access latency. We observe that threads running on different SMs share 55% of the data they read from the memory. Unfortunately, as the L1Ds are in the non-coherent memory domain, each SM independently fetches data from the L2 or the off-chip memory into its L1D, even though the data may be currently available in the L1D of another SM. Our goal is to service L1D read misses via other SMs, as much as possible, to cut down costly accesses to the L2 or the off-chip DRAM. To this end, we propose a new data sharing mechanism, called Cross-Core Data Sharing (CCDS). CCDS employs a predictor to estimate whether or not the required cache block exists in another SM. If the block is predicted to exist in another SM’s L1D, CCDS fetches the data from the L1D that contains the block. Our experiments on a suite of 26 workloads show that CCDS improves average energy and performance by 1.30 × and 1.20 ×, respectively, compared to the baseline GPU. Compared to the state-of-the-art data-sharing mechanism, CCDS improves average energy and performance by 1.37 × and 1.11 ×, respectively.

图形处理单元 (GPU) 是各种应用领域的首选加速器，因为它们可以加速大规模并行工作负载，并且可以使用 CUDA 和 OpenCL 等通用编程框架轻松进行编程。每个流多处理器 (SM) 都包含一个 L1 数据缓存 (L1D)，以利用数据访问的局部性。由于两个原因，L1D 未命中对于 GPU 来说代价高昂。首先，L1D 缺失会消耗大量能量，因为在 L2 缺失的情况下，它们需要通过片上网络访问 L2 缓存 (L2) 和片外 DRAM。其次，如果 GPU 没有足够的活动扭曲来隐藏较长的内存访问延迟，则 L1D 未命中会带来性能开销。我们观察到，运行在不同 SM 上的线程共享从内存读取的数据的 55%。不幸的是，由于 L1D 位于非一致存储器域中，每个 SM 独立地将数据从 L2 或片外存储器提取到其 L1D 中，即使该数据当前可能在另一个 SM 的 L1D 中可用。我们的目标是尽可能通过其他 SM 解决 L1D 读取未命中的问题，以减少对 L2 或片外 DRAM 的昂贵访问。为此，我们提出了一种新的数据共享机制，称为跨核心数据共享（CCDS）。CCDS使用预测器来估计所需的缓存块是否存在于另一个 SM 中。如果预测该块存在于另一个SM的L1D中，则CCDS从包含该块的L1D中获取数据。我们对一组 26 个工作负载进行的实验表明，与基准 GPU 相比， CCDS的平均能耗和性能分别提高了 1.30 倍和 1.20 倍。与最先进的数据共享机制相比，CCDS的平均能量和性能分别提高了 1.37 倍和 1.11 倍。