R&D: HyperDB, Novel Key Value Store for Reducing Background Traffic in Heterogeneous SSD Storage

ACM Digital Library has published, in ICPP ’24: Proceedings of the 53rd International Conference on Parallel Processing, an article written by Ruisong Zhou, Yuzhan Zhang, Chunhua Li, Ke Zhou, Huazhong University of Science and Technology, China, Peng Wang, Theory Lab, Huawei Hong Kong Research Center, Huawei Technologies Co., Ltd., Hong Kong, Gong Zhang, Huawei Technologies Co., Ltd., Hong Kong, Ji Zhang, Huawei Technologies Co., Ltd., Switzerland, and Guangyu Zhang, Huawei Technologies Co., Ltd., China.

Abstract: “Log-structured merge tree (LSM-tree) has been widely adopted by modern key-value stores. Deploying LSM-tree across heterogeneous SSD storage which combines the fast but expensive NVMe storage tier with the slow but economical SATA storage tier has emerged as the optimal choice for maximizing cost-effectiveness. However, existing studies typically focus on optimizing the performance of individual storage layers, thereby impeding the full utilization potential of both storage layers. We notice that they tend to over-rely on one storage layer and underutilize the other. In this paper, we present HyperDB, a novel hybrid key-value store designed to enhance the overall performance of both layers via deploying tailored data structures in different media. Especially, HyperDB devises a zone-based data layout for NVMe SSDs to reduce migration overhead, while also implementing a semi-sorted table on the SATA storage layer to minimize merge overhead. Furthermore, we propose a preemptive compaction method at the block-granularity level to further alleviate resource consumption caused by background compaction. Experimental results show that HyperDB achieves 2.25 × faster on average throughput and a 60.3% reduction in background task traffic, compared to the standard use of RocksDB in data centers today.“