Memory tiering is an effective strategy for enhancing the memory-to-core ratio in modern compute servers. Intel’s Flat Memory Mode (FLAT) represents a cutting-edge, hardware-managed memory tiering system for CXL memory, operating at cache-line (64B) granularity with yet unknown potential for database management systems. In this paper, we evaluate the performance of FLAT using OLAP workloads in SAP HANA, an in-memory database management system. Our findings demonstrate that flat memory outperforms directly attached CXL memory by leveraging data placement and exploiting locality. However, FLAT currently accepts only a 1:1 ratio between local and CXL memory. When the local memory capacity increases, FLAT operates in a mixed mode (MIXED) containing flat memory and additional dedicated memory with remaining local memory in a separate NUMA node. Our evaluation of MIXED shows that performance degradation depends on access patterns and the amount of exploited data locality when moving table data to flat memory or allocating heap memory in flat memory. To address the configuration complexity of two separate NUMA nodes in MIXED, we propose a new CONSOLIDATED mode that merges two NUMA nodes. Our evaluation shows that the consolidated mode results in negligible performance degradation, regardless of the ratio between dedicated memory and flat memory, and requires no software modifications at the application level.
This paper introduces G-ALP, a GPU-optimized version of ALP, which is a recent and state-of-the-art compression scheme for floating-point. This GPU-optimization is based on two core ideas. First, all parts of the decoding process must be fully data-parallelized. In this paper, we fully data-parallelize exception patching, which typically applies to only 1% of the data. While patching has negligible performance cost on CPUs, it can become the main bottleneck on GPUs if it is not data-parallel. Second, the decoding API must minimize its register footprint, a highly scarce resource on GPUs, and hence deliver just one value-at-a-time. Our unique aim is to integrate G-ALP decoding into GPU kernels that consume data from global memory, rather than let decompression be a separate kernel. We consider these two ideas general guidelines for future GPU-optimized lightweight encodings, and a significant evolution of our new FastLanes file format, making it GPU-friendly. We extensively test G-ALP in a series of microbenchmarks and evaluate its performance on an NVIDIA V100 GPU and an NVIDIA RTX4070 Super Ti GPU, demonstrating superior performance compared to NVIDIA nvCOMP and ndzip in both decoding and filtering queries.
GPU-accelerated database systems promise significant performance gains for analytical workloads, but their adoption remains limited due to fundamental data movement bottlenecks when the dataset does not fit in GPU memory. Compression can significantly accelerate data movement, but when the dataset does not fit in GPU memory, the choice of compression algorithm (lightweight or heavyweight) can have a wide range of effects, including a potential slowdown compared to uncompressed data. This paper analyzes the impact of compression on query execution performance for larger-than-GPU-memory datasets stored on arrays of NVMe SSDs. We integrate pipelined data movement and on-GPU decompression into a GPU query processing engine. Our experiments show that when storage bandwidth is limited, compression always accelerates query execution, with lightweight compression incurring a higher speedup than heavyweight compression. As storage bandwidth is increased, however, compression can incur a speedup or a slowdown, depending on the processing intensity of the query. We quantify the impact of compression on query execution time with a microbenchmark that varies both storage bandwidth as well as the selectivity and number of joins. We find that heavyweight compression suffers from performance degradation with less processing-intensive queries and lower storage bandwidths than lightweight compression. We further analyze the compression algorithm’s impact on GPU utilization using an extended roofline model incorporating the SSD-to-GPU bandwidth bottleneck. For queries from the Star Schema Benchmark, the results show that while lightweight compression drastically increases the count of executed instructions, the improvement in effective storage bandwidth results in improved query execution times.
As hardware complexity increases, profiling becomes essential for understanding system behavior. This paper compares different x86 sampling implementations for memory access profiling, revealing their complementary capabilities and limitations. Plus, we demonstrate that current abstractions like the perf subsystem inadequately expose platform-specific features.
We present insert-optimized implementations of three fundamental data sketching algorithms: Count Sketch (CS), SpaceSaving (SS), and Karnin-Lang-Liberty (KLL). While these sketches are widely used for approximate query processing and stream analytics, their practical insert performance often falls short of their full potential. Through careful engineering and novel implementation strategies, we achieve substantial throughput improvements over both naive and existing implementations. Our approach demonstrates speedups of up to 12.30x, 2.00x, and 1.52x for CS, SS, and KLL respectively when compared to the industry-standard Apache DataSketches library. When measured against naive implementations, we achieve even more dramatic improvements: 8.63x for CS, 7.03x for SS, and 446.00x for KLL. We also measure against available open-source implementations used as baselines in other works and outperform them by a large margin. We detail the technical optimizations enabling these improvements, including fast hash range reduction and hash sharing for Count Sketch, SIMD vectorization for SpaceSaving, and preallocation and branching improvements for Karnin-Lang-Liberty. Our implementations maintain the theoretical guarantees of the original algorithms while providing substantially better practical insert performance, making them particularly valuable for high-throughput streaming applications where update speed is critical.
Memory can be located close to a CPU, at remote sockets, or on devices connected via interconnects such as CXL or NVLink. A larger distance between memory and a core accessing the memory usually results in higher access latency. Software prefetching algorithms claim to hide memory access latencies by moving data to the CPU cache before a core accesses the data. In this work, we analyze to what extent software prefetching can hide increased memory access latencies. We evaluate these on seven systems, each offering different memory technologies and access latencies. We show that prefetching can increase performance by up to 2.6x and 2.8x for B+-Tree and binary search workloads. We find that CPU fill buffers, which track L1 cache misses, and a workload's memory intensity dictate how much access latency can be hidden. CPUs implement prefetches differently. We introduce microbenchmarks that identify concrete target cache and eviction strategies for different prefetch localities across x86 and ARM architectures. When the fill buffers are full, CPUs either drop prefetches or halt until all can be executed. We refer to these behaviors as weak and strong prefetching reliability. We introduce microbenchmarks identifying a CPU's reliability. When prefetching 8 KiB B+-Tree nodes, weak reliability achieves a speedup of 2x while strong reliability degrades performance with a slowdown of 2.5x for lookup workloads.
Accelerators allow computation to be offloaded from the CPU and improve query performance. Intel has recently introduced a dedicated Data Accelerator Complex (DAC) for its scalable Xeon processor family. Among the four accelerators, the In-Memory Analytics Accelerator (IAA) provides functionality for offloading query processing operations from the CPU. In this work, we investigate the potential of this accelerator for database queries. We propose a coreless query processing approach in which query operators such as scan, filter, but also sorting, and joins are executed on the accelerator. Our evaluation shows that offloading tasks such as filter and join operators to IAA achieves performance comparable to high-thread-count CPU processing. Further speedups on compressed data enable the DBMS to free CPU resources more efficiently for other tasks.
Content-based page sharing (de-duplication) is a heavily used technique to improve memory efficiency in virtualized systems by identifying and merging identical pages. For many years now, the Linux kernel has offered this de-duplication technique via the Kernel Samepage Merging (KSM) feature. Although KSM in general works well, it is not used in multi-tenant database systems even though multiple tenants often manage similar data. One reason is that pages must be binary identical, which is a severe restriction. Secondly, KSM is seemingly scheduled as a single-threaded process by the OS, independently of the database workload, which further limits its applicability for in-memory systems with terabytes of main memory. To overcome that, we propose an extended de-duplication mechanism called De2Dup for memory-centric multi-tenant database engines. De2Dup extends de-duplication with a delta mechanism to significantly boost the application, especially when pages are not binary identical. Moreover, our De2Dup mechanism allows to steer the search for duplicates and has low overhead as we are able to offload the complete execution to a modern on-chip accelerator for memory operations in an asynchronous manner on recent Intel server processors. In addition, De2Dup offers an efficient way for on-the-fly tenant-aware data reconstruction during scan operations.
The process of training and serving deep learning (DL) models is computationally expensive, mandating the use of powerful and expensive accelerators such as GPUs and TPUs. Furthermore, the prevalence of GPUs in data centers today motivate developing database systems that can leverage the available GPU resources. Both the latency of DL tasks and database queries and high utilization of these accelerators depend on how efficiently we can move the data to the accelerators. Given today’s dataset sizes, fitting everything in GPU or even CPU memory is not always feasible or can be expensive. The I/O path while fetching the data from disks, however, still dominantly relies on CPUs. In this work, we take a step toward understanding today’s landscape for optimizing the I/O path for reading data to GPUs from disks, with a focus on SSDs. First, we review the prominent technologies that target GPU-centric storage accesses. Then, we dive deeper into BaM [38], as the state-of-the-art method for GPU-centric storage, and evaluate its performance in comparison to the state-of-the-art CPU-centric storage interface SPDK. Our results demonstrate that while BaM is able to match the performance of SPDK without involving CPUs on the I/O path, this comes at the cost of a very high GPU use. Finally, we highlight future research directions to enable an I/O path that is both efficient and easy-to-adopt for data-intensive systems that use GPUs.
Cloud databases have become prevalent, as evidenced by the rapid growth of systems such as BigQuery, Snowflake, and Databricks. Concurrently, there has been a significant increase in the requirements for secure data processing when outsourcing databases to the cloud. For this, Trusted Execution Environments (TEEs) have emerged as a key technology in the cloud, which is witnessed by the fact that all cloud providers offer TEEs in their service portfolios. However, Amazon Web Services’ (AWS) approach to TEEs based on Nitro Enclaves fundamentally differs from that of other cloud providers like Microsoft and Google or standard technologies such as Intel SGX. In this paper, we thus set out the goal to understand the implications of using AWS Nitro Enclaves for cloud databases. Although Nitro Enclaves initially appear to be a promising platform for pure TEE performance, they come with significant limitations regarding communication with the Nitro Enclave. Our benchmark results provide insight into the performance and practical challenges of deploying database workloads in AWS Nitro Enclaves, offering valuable guidance for practitioners and researchers.
Cross-modal Approximate Nearest Neighbor Search (ANNS) is crucial for a growing number of applications, including search engines and recommendation systems. However, existing vector search indexes often struggle with either poor search efficiency or slow index construction when handling cross-modal ANNS queries. To overcome these limitations, we introduce ParaGraph, a CPU-GPU co-processing system that enhances both search efficiency and construction speed for cross-modal ANNS. ParaGraph employs novel multi-round top-.. projection and batched search-and-refine techniques for index construction. Additionally, it leverages modern heterogeneous hardware architectures by distributing computationally distinct tasks across the GPU and CPU, augmented with in-depth optimizations to maximize parallelism and performance. Compared to the state-of-the-art cross-modal ANNS index, Para-Graph achieves a 4.1x to 4.9x speedup in index construction and a 50% reduction in index size, while maintaining search efficiency.
Kernel-bypass technologies eliminate the overhead of traditional OS stacks, offering direct access to high-speed I/O devices such as network and storage. This paper argues that kernel-bypass is no longer an optional optimization but a necessary architectural strategy for I/O-heavy applications like database systems. The motivation stems from two trends: stagnating CPU performance and rapid advances in I/O hardware, such as 800 Gbit/s NICs and SSDs exceeding 12M IOPS. In our evaluation, we show that, given these trends, it is no longer possible for DBMSs to saturate modern NICs or SSDs with traditional kernel stacks. We thus urge the research community to prioritize kernel-bypass technologies to fully harness the potential of emerging hardware in database systems.
The performance of modern Big Data systems used for data processing is often bottlenecked by data movement across various components. Offloading part of the processing closer to the storage and network to reduce this bottleneck is a compelling idea, and in today’s hardware landscape, there are many different types of Smart Storage, Smart NIC, or Smart Switch devices one could choose from. One challenge, however, is that it is often unclear at design time what improvements of the end workload one can achieve with a given hardware. Co-design is typically mentioned in related work as the solution, but it is far from obvious what this entails in practice: what information about the hardware, software system, and workload is taken into account for decision-making is often implicit in related work. In this work, we propose a model-driven methodology for right-sizing offloading to benefit an end workload. Our methodology determines the target processing rate an offload device should have in a specific data processing system, without over-fitting. This helps designers pick the right hardware for offloading. Our methodology relies on modeling the system as a network of queues, based on different levels of information about the system, which allows determining the general usefulness of offload and the specific benefits to a workload of interest. We demonstrate how our methodology avoids under- or over-provisioning offload devices in a case study.
Vector databases have emerged as a new type of systems that support efficient querying of high-dimensional vectors. Many of these offer their database as a service in the cloud. However, the variety of available CPUs and the lack of vector search benchmarks across CPUs make it difficult for users to choose one. In this study, we show that CPU microarchitectures available in the cloud perform significantly differently across vector search scenarios. For instance, in an IVF index on float32 vectors, AMD's Zen4 gives almost 3x more queries per second (QPS) compared to Intel's Sapphire Rapids, but for HNSW indexes, the tables turn. However, when looking at the number of queries per dollar (QP$), Graviton3 is the best option for most indexes and quantization settings, even over Graviton4 (Table 1). With this work, we hope to guide users in getting the best "bang for the buck" when deploying vector search systems.
We provide an evaluation of an analytical workload in a confidential computing environment, combining DuckDB with two technologies: modular columnar encryption in Parquet files (data at rest) and the newest version of the Intel SGX Trusted Execution Environment (TEE), providing a hardware enclave where data in flight can be (more) securely decrypted and processed. One finding is that the "performance tax" for such confidential analytical processing is acceptable compared to not using these technologies. We eventually manage to run TPC-H SF30 with under 2x overhead compared to non-encrypted, non-enclave execution; we show that, specifically, columnar compression and encryption are a good combination. Our second finding consists of dos and don'ts to tune DuckDB to work effectively in this environment. There are various performance hazards: potentially 5x higher cache miss costs due to memory encryption inside the enclave, NUMA penalties, and highly elevated cost of swapping pages in and out of the enclave -- which is also triggered indirectly by using a non-SGX-aware malloc library.
The rapid growth of machine learning capabilities and the adoption of data processing methods using vector embeddings sparked a great interest in creating systems for vector data management. While the predominant approach of vector data management is to use specialized index structures for fast search over the entirety of the vector embeddings, once combined with other (meta)data, the search queries can also become selective on relational attributes - typical for analytical queries. As using vector indexes differs from traditional relational data access, we revisit and analyze alternative access paths for efficient mixed vector-relational search. We first evaluate the accurate but exhaustive scan-based search and propose hardware optimizations and alternative tensor-based formulation and batching to offset the cost. We outline the complex access-path design space, primarily driven by relational selectivity, and the decisions to consider when selecting an exhaustive scan-based search against an approximate index-based approach. Since the vector index primarily avoids expensive computation across the entire dataset, contrary to the common relational knowledge, it is better to scan at lower selectivity and probe at higher, with a cross-point between the two approaches dictated by data dimensionality and the number of concurrent search queries.
This paper presents Seamless, a switch-based accelerator for disaggregated SSD-based systems. Seamless comprises two fundamental components: (1) a hardware-accelerated, unified remote storage protocol that ensures efficient data access to Flash and remote memory. (2) A hardware-accelerated concurrency protocol with a simple interface like a buffer manager: fix(page_id, {exclusive, shared}) for acquiring page latches and unfix(page_id) for releasing them. Seamless achieves this by using a P4-programmable switch. To enable transparent storage access and caching, we maintain each page's location within the switch, whether in memory or on an SSD. Upon receiving a page request, Seamless determines the page's location and rewrites the communication protocol at line rate to NVMe-oF or RDMA accordingly. We manage the latches directly on the switch to maintain consistency and provide concurrency control. We show initial promising results for switch-based acceleration: Seamless hard-level accelerated protocol outperforms a software-based RDMA-optimized system by 20%.
The ubiquity of variable-length integers in data storage and communication necessitates efficient decoding techniques. In this paper, we present SFVInt, a simple and fast approach to decode the prevalent Little Endian Base-128 (LEB128) varints. Our approach effectively utilizes the Bit Manipulation Instruction Set 2 (BMI2) in modern Intel and AMD processors, achieving significant performance improvement while maintaining simplicity and avoiding overengineering. SFVInt, with its generic design, effectively processes both 32-bit and 64-bit unsigned integers using a unified code template, marking a significant leap forward in varint decoding efficiency. We thoroughly evaluate SFVInt's performance across various datasets and scenarios, demonstrating that it achieves up to a 2x increase in decoding speed when compared to varint decoding methods used in established frameworks like Facebook Folly and Google Protobuf.
We show that compression can be a win-win for GPU data processing: it not only allows to store more data in GPU global memory, but can also accelerate data processing. We show that the complete redesign of compressed columnar storage in FastLanes, with its fully data-parallel bit-packing and encodings, also benefits GPU hardware. We micro-benchmark the performance of FastLanes on two GPU architectures (Nvidia T4 and V100) and integrate FastLanes in the Crystal GPU query processing prototype. Our experiments show that FastLanes decompression significantly outperforms previous decompression methods in micro-benchmarks, and can make end-to-end SSB queries up to twice faster compared to uncompressed query processing - in contrast to previous work where GPU decompression caused execution to slow down. We further discovered that an access granularity of decoding vectors of 1024 values is too large for a single GPU warp due to register pressure. We mitigate this here using mini-vectors - a future work question is how to further reduce this granularity with minimal impact on efficiency.
