当前位置: X-MOL 学术VLDB J. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Towards flexibility and robustness of LSM trees
The VLDB Journal ( IF 4.2 ) Pub Date : 2024-01-11 , DOI: 10.1007/s00778-023-00826-9
Andy Huynh , Harshal A. Chaudhari , Evimaria Terzi , Manos Athanassoulis

Log-structured merge trees (LSM trees) are increasingly used as part of the storage engine behind several data systems, and are frequently deployed in the cloud. As the number of applications relying on LSM-based storage backends increases, the problem of performance tuning of LSM trees receives increasing attention. We consider both nominal tunings—where workload and execution environment are accurately known a priori—and robust tunings—which consider uncertainty in the workload knowledge. This type of workload uncertainty is common in modern applications, notably in shared infrastructure environments like the public cloud. To address this problem, we introduce Endure, a new paradigm for tuning LSM trees in the presence of workload uncertainty. Specifically, we focus on the impact of the choice of compaction policy, size ratio, and memory allocation on the overall performance. Endure considers a robust formulation of the throughput maximization problem and recommends a tuning that offers near-optimal throughput when the executed workload is not the same, instead in a neighborhood of the expected workload. Additionally, we explore the robustness of flexible LSM designs by proposing a new unified design called K-LSM that encompasses existing designs. We deploy our robust tuning system, Endure, on a state-of-the-art key-value store, RocksDB, and demonstrate throughput improvements of up to 5\(\times \) in the presence of uncertainty. Our results indicate that the tunings obtained by Endure are more robust than tunings obtained under our expanded LSM design space. This indicates that robustness may not be inherent to a design, instead, it is an outcome of a tuning process that explicitly accounts for uncertainty.



中文翻译:

实现 LSM 树的灵活性和鲁棒性

日志结构合并树(LSM 树)越来越多地用作多个数据系统背后的存储引擎的一部分,并且经常部署在云中。随着依赖基于LSM的存储后端的应用程序数量的增加,LSM树的性能调优问题越来越受到关注。我们考虑名义调整(其中工作负载和执行环境是先验准确已知的)和鲁棒调整(考虑工作负载知识的不确定性)。这种类型的工作负载不确定性在现代应用程序中很常见,特别是在公共云等共享基础设施环境中。为了解决这个问题,我们引入了Endure,这是一种在工作负载不确定的情况下调整 LSM 树的新范例。具体来说,我们关注压缩策略的选择、大小比例和内存分配对整体性能的影响。Endure考虑了吞吐量最大化问题的稳健公式,并建议进行调整,以便在执行的工作负载不相同时(而不是在预期工作负载的附近)提供接近最佳的吞吐量。此外,我们通过提出一种名为 K-LSM 的新统一设计来探索灵活 LSM 设计的稳健性,该设计包含现有设计。我们在最先进的键值存储 RocksDB 上部署了强大的调优系统Endure ,并在存在不确定性的情况下展示了高达 5倍的吞吐量改进。我们的结果表明, Endure获得的调整比我们扩展的 LSM 设计空间下获得的调整更稳健。这表明鲁棒性可能不是设计所固有的,而是明确考虑不确定性的调整过程的结果。

更新日期:2024-01-11
down
wechat
bug