An increasing number of high-performance computers are equipped with high bandwidth memory. Their CPUs have cache-coherent access to this memory type over high-speed interconnects or as unified memory. We analyse two such systems, NVIDIA’s GH200 and AMD’s MI300A. We discuss their memory setup and measure memory access latency and throughput. We evaluate weighted memory page interleaving and strided memory access as candidates for increasing the measured throughput. Weighted interleaving utilises CPU and GPU memory simultaneously and enables exceeding the measured throughput on only CPU or GPU memory for all tested workloads. Using strided memory access, which accesses memory at fixed intervals, we achieve higher throughput than using a non-strided sequential pattern.
The AI hardware boom has driven modern data centers toward HPC-style architectures centered on GPU clusters, RDMA-capable networks, and high-throughput NVMe storage. While designed primarily for training and inference, this infrastructure also creates new opportunities for designing the next generation of scalable database systems for analytical workloads on top of such data centers. In particular, the combination of GPU-centric computation, fast networking, and fast storage enables disaggregated architectures that extend beyond single-node, GPU-memory-resident execution. This paper discusses the challenges and design considerations of analytical query processing on such disaggregated GPU-centric systems. We examine how modern networks and storage enable distributed execution and out-of-memory processing, and how their interaction shapes end-to-end performance. Our recent results show that naïve use of existing I/O abstractions can underutilize both compute and I/O bandwidth due to insufficient overlap between computation and data movement. We therefore identify effective I/O-computation overlap as a key requirement for fully exploiting the AI data center stack, and outline future research directions for next-generation analytical database architectures.
Buffer pool translation—resolving an on-disk page identifier to an in-memory frame—was once a heavyweight operation whose bookkeeping consumed a substantial fraction of CPU cycles in early OLTP engines. Recent systems reduce this cost with techniques such as pointer swizzling, in-page hints, OS page-table mappings, prediction, sharding, and direct arrays. This paper surveys buffer pool translation mechanisms and then studies two orthogonal ways to reduce the cost of hash-table-based translation: placing related pages near each other in the frame array so neighboring pages prefer neighboring frame slots, and choosing a lookup policy that bypasses the hash table when a predicted frame validates. The benefit of each depends on workload locality, displacement, and payload size: on an in-memory translation microbenchmark, combining relation-local placement with bypass yields up to 1.46 × higher translation throughput, while the advantage shrinks to near zero once large payload reads dominate per-access cost.
Durch die enormen Datenmengen, die moderne Computersysteme effizient verarbeiten müssen, rückt der Hauptspeicher immer weiter in das Zentrum der Betrachtung bei der Softwareentwicklung, bei Herstellern von Hardware und in der Wissenschaft. Neue Technologien wie HBM („High-Bandwidth Memory“), PIM/PNM („Processing In/Near Memory“) und persistente Hauptspeichermodule werden entwickelt und in komplexen NUMA-Topologien, zum Beispiel auf Basis von CXL, verknüpft. All dies hat Auswirkungen auf den kompletten Systemsoftwarestapel und bis in die Anwendungen hinein. In diesem Sinne sind viele neuartige Hauptspeichertechnologien disruptiv. In dieser Lage ist das Ziel des im Jahr 2022 von der DFG eingerichteten Schwerpunktprogramms 2377 die objektive Bewertung dieser neuen Technologien und die Erforschung der Frage, wie sich diese auf bestehende und gänzlich neue Hardware‑/Software-Architekturen auswirken. Dabei entwickeln die Teilprojekte, die verschiedenen Forschungs-Communitys der (Kern‑)Informatik zuzurechnen sind, aktiv neuartige Lösungen, um die Potenziale der Technologien aufzuzeigen. Dieser Artikel erläutert die Hintergründe und Ziele des Programms, fasst die Ergebnisse der dreijährigen ersten Förderperiode zusammen und gibt einen Ausblick auf die zweite Förderperiode.
Traditionally, deploying database systems required acquiring and managing large, expensive servers on-premise. These servers feature a memory and storage hierarchy, including CPU caches, DRAM memory, HDDs and/or SSDs, and sometimes tape. For almost 40 years, Gray and Putzolu’s five-minute rule [4, 10, 12, 13] has helped guide system architects to the break-even point between memory caching and direct local storage access for these systems. The rise of cloud computing has significantly changed the way systems are deployed. Cloud services offer on-demand computing and storage, removing the need for companies to acquire and manage their own hardware. Furthermore, the cloud has introduced a new layer of the storage hierarchy for database systems: object stores. This durability and low cost have made object stores ubiquitous in modern cloud-native databases and data infrastructures [5, 6, 16]. We believe a rule of thumb similar to the five-minute rule is needed for object caches and storage for disaggregated cloud data system designs. However, it is not straightforward to adapt the established rules to the cloud as they presume fixed hardware, while, in the cloud, resources are dynamic and costs are determined by usage. One can argue that accessing object stores over the network incurs higher latency and lower bandwidth than directly attached storage devices, placing them at the bottom of the storage hierarchy. However, advances in cloud storage and networking have significantly impacted cost-effectiveness, latency variability, and dynamic workload optimization, challenging these traditional assumptions. For example, in non-storage-optimized AWS instances, network bandwidth often exceeds local storage read bandwidth, and writing to local storage devices tends to be half as fast as reading, while writing to S3 can match network bandwidth. This paper is a summary of our earlier work work [9] that proposes a cost model and new rules of thumb to help system designers determine when caches become cost-effective for analytical workloads in the cloud. While perhaps unsurprising, our analysis on AWS shows that caches are beneficial when a system makes (1) two requests per hour for latency-sensitive workloads, or (2) seven requests per second for non-latency-sensitive workloads. These results are consistent with and help explain the near ubiquity of object store caches in cloud analytics systems.
