Traditional ARIES has long been the standard for logging and recovery in database systems. Yet, its overhead limits system efficiency on modern hardware, leading to a shift towards decentralized logging for improved performance and scalability. Decentralized logging, however, poses two major challenges: (1) its integration with existing commit processing protocols fails to provide low-latency commits, (2) current variants introduce many critical trade-offs that hinder widespread adoption. This paper aims to address these gaps. To tackle the first issue, we introduce Autonomous Commit , the first commit protocol that fully utilizes modern NVMe SSDs to achieve low-latency durable commits with decentralized logging. Second, we propose Commit Causality Clocks , a new dependency tracking mechanism optimized for decentralized logging that retains the strengths of existing approaches while eliminating their limitations. We also revisit the recovery subsystem, enabling DBMS logging to work directly on block devices for optimal performance during normal operation. Our experimental evaluation demonstrates that these techniques together deliver very good scalability and low-latency durable commits across diverse workloads, enabling a high-performance logging and recovery design.
We propose monotonic consistent caching (MCC), a cache scheme for applications that demand transactional guarantees. MCC warrants that a transaction-like request always sees a consistent view of the backend database and that observed writes over the cache will not be lost, even if it operates on conventional cache systems, e.g., Memcached or Redis, that come without such guarantees. Unlike traditional caching which is trivially in Ptime under the offline (batch) model where requests are known in advance, we show that the complexity of MCC ranges from Ptime to Np-Complete , depending on the version selection strategy in case of cache hits that violate transactional guarantees. We characterize MCC via a notion of obsolete items, based on which we abstract a principle for designing competitive MCC policies. By applying the principle, we develop optimal and approximate MCC policies for the batch model, where requests in a batch are known in advance. For the online and semi-online models, we develop ML-augmented policies that benefit from blackbox ML models for classifying obsolete items, while being provably competitive even if the ML is arbitrarily bad. We further implement a pluggable system that supports all proposed MCC policies. Using benchmark and real-life traces, we show that MCC policies reduce 34.9% of database reads for Redis atop HBase and improve their throughput by 59.1%.
Index is an important component in database systems. Learned indices have been shown to outperform traditional tree-based index structures for fixed-sized integer or floating point keys. However, the application of the learned solution to variable-length string keys is under-researched. Our experiments show that existing learned indices for strings fail to outperform traditional string indices, such as HOT and ART. String keys are long and variable sized, and often contain skewed prefixes, which make the last-mile search expensive, and adversely impact the capability of learned models to capture the skewed distribution of string keys. In this paper, we propose a novel learned index for string keys, LITS ( L earned I ndex with Hash-enhanced Prefix T able and S ub-tries). We design an optimized learned model that combines a global Hash-enhanced Prefix Table (HPT) with per-node local linear models to better distinguish string keys. Moreover, we exploit two techniques, i.e., dynamic scale factor and cell-based root, to improve the space efficiency. We prove that the resulting design is guaranteed to have O ( N ) space cost. Furthermore, we leverage compact leaf nodes and hybrid structures with a PMSS model to support efficient point and range queries. Our experimental results using eleven string data sets show that LITS achieves up to 2.19 × and 1.85 × improvement over HOT and ART for point operations with comparable scan performance and space cost.
The incorporation of domain knowledge (DK) in AI has been studied for years and turns out to be critical in practice. Functions are also a basic notion for dealing with data science projects and are somehow related to DK. Consider the following scenario. Let \(D(y, z_1, \ldots , z_n)\) be a dataset, Alice a data scientist, Bob a domain expert and \(y = f(z_1, \ldots , z_n)\) a function known to Bob from his background knowledge. Alice is interested in the following simple yet crucial questions: How to define the satisfaction of f in D ? How to measure that satisfaction efficiently? How does this satisfaction relate to the supervised learning task of learning f from D ? It turns out that these problems are related to the study of counterexamples through the use of functional dependencies (FDs) and, in particular, FD measures used to quantify their satisfaction in a dataset such as the \(g_3\) indicator where the equality is replaced by more flexible predicates. In this article, we first examine the complexity of computing \(g_3\) . It is known that \(g_3\) can be computed in polynomial time when using equality, while it becomes NP -hard when using general predicates. Our goal is to refine this dichotomy by studying the impact of the following common properties: reflexivity, transitivity, symmetry, and antisymmetry. We show that symmetry and transitivity together are sufficient to guarantee that the \(g_3\) can be computed in polynomial time. However, removing one of them makes the problem NP -hard. Second, we study the computation of \(g_3\) in the polynomial and NP -hard cases identified previously. We propose different exact and approximate solutions for the computation of \(g_3\) in both cases. We compare these solutions in a detailed experimental study of time performance and approximation accuracy. All the algorithms are also made available via fastg3 , an open-source Python library with an underlying C++ implementation. Finally, we link counterexamples and \(g_3\) to supervised learning with a web application called adesit . adesit is intended to be part of an iterative data refinement process right after data selection and just before the machine learning process itself. It provides a way to evaluate the ability of a dataset to perform well for a given supervised learning problem through statistical and visual exploration. In a last section, we validate our approach by applying it to the industrial problem of air gap monitoring in compact hydro-generators.
A data structure is history independent if its internal representation reveals nothing about the history of operations beyond what can be determined from the current contents of the data structure. History independence is typically viewed as a security or privacy guarantee, with the intent being to minimize risks incurred by a security breach or audit. Despite widespread advances in history independence, there is an important data-structural primitive that previous work has been unable to replace with an equivalent history-independent alternative— dynamic partitioning . In dynamic partitioning, we are given a dynamic set S of ordered elements and a size-parameter B , and the objective is to maintain a partition of S into ordered groups, each of size Θ ( B ). Dynamic partitioning is important throughout computer science, with applications to B-tree rebalancing, write-optimized dictionaries, log-structured merge trees, other external-memory indexes, geometric and spatial data structures, cache-oblivious data structures, and order-maintenance data structures. The lack of a history-independent dynamic-partitioning primitive has meant that designers of history-independent data structures have had to resort to complex alternatives. In this paper, we achieve history-independent dynamic partitioning. Our algorithm runs asymptotically optimally against an oblivious adversary, processing each insert/delete with O (1) operations in expectation and O ( B log N /log logN ) with high probability in set size N . We also use our dynamic partitioning scheme to build a history-independent B -tree, history-independent fusion tree, and external-memory skip list.
