State machine replication (SMR) solutions underpin many modern distributeddatabase systems. They provide redundancy through replication and ensure these systems remain available and consistent even in the face of failures. This Foundations and Trends article explores various SMR solutions built upon foundational consensus SMR protocols, such as Viewstamped Replication, Paxos, and Raft, and examines how they solve the core challenges of operation distribution and sequencing. It categorizes SMR designs into diverse architectural styles (including single-leader, sequential, leaderless, and multi-leader approaches) and details the latency and throughput trade-offs for each. The article also details specific protocol optimizations, such as flexible quorums, lease-based local reads, and hardware acceleration using RDMA or in-network programmable smart switches. Beyond theoretical concepts like the CAP theorem and the FLP impossibility, the article connects these algorithmic designs directly to their real-world deployments, including control-plane systems like ZooKeeper, NoSQL data stores like MongoDB, and NewSQL distributed databases like Google Spanner and CockroachDB. This comprehensive review provides system designers with a detailed taxonomy of SMR traits to navigate the landscape of fault-tolerant distributed replication.
Unstructured data, in the form of text, images, video, and audio, is produced at exponentially higher rates. In tandem, machine learning (ML) methods have become increasingly powerful at analyzing unstructured data. Modern ML methods can now detect objects in images, understand actions in videos, and even classify complex legal texts based on legal intent. Combined, these trends make it increasingly feasible for analysts and researchers to automatically understand the "real world." However, there are major challenges in deploying these techniques: 1) executing queries efficiently given the expense of ML methods, 2) expressing queries over bespoke forms of data, and 3) handling errors in ML methods. In this monograph, we discuss challenges and advances in data management systems for unstructured data using ML, with a particular focus on video analytics. Using ML to answer queries introduces new challenges. First, even turning user intent into queries can be challenging: it is not obvious how to express a query of the form "select instances of cars turning left." Second, ML models can be orders of magnitude more expensive compared processing traditional structured data. Third, ML models and the methods to accelerate analytics with ML models can be error-prone. Recent work in the data management community has aimed to address all of these challenges. Users can now express queries via user-defined functions, opaquely through stan-dard structured schemas, and even by providing examples. Given a query, recent work focuses on optimizing queries by approximating expensive "gold" methods with varying levels of guarantees. Finally, to handle errors in ML models, recent work has focused on applying outlier and drift detection to data analytics with ML.
The performance of a query crucially depends on the ability of the query optimizer to choose a good execution plan from a large space of alternatives. With the discovery of algebraic transformation rules and the emergence of new application-specific contexts, extensibility has become a key requirement for query optimizers. This monograph describes extensible query optimizers in detail, focusing on the Volcano/Cascades framework used by several database systems including Microsoft SQL Server. We explain the need for extensible query optimizer architectures and how the optimizer navigates the search space efficiently. We then discuss several important transformations that are commonly used in practice. We describe cost estimation, an essential component that the optimizer relies upon to quantitatively compare alternative plans in the search space. We discuss how database systems manage plans over their lifetime as data and workloads change. We conclude with a few open challenges.
In an era of increasingly interconnected information, graph-structured data has become pervasive across numerous domains from social media platforms and telecommunication networks to biological systems and knowledge graphs. However, traditional database management systems often struggle when confronted with the unique challenges posed by graph-structured data, in large part due to the explosion of intermediate results, the complexity of join-heavy queries, and the use of regular path queries. This survey provides a comprehensive overview of modern query processing techniques designed to address these challenges. We focus on four key components that have emerged as pivotal in optimizing queries on graph-structured databases: (1) Predefined joins, which leverage precomputed data structures to accelerate joins; (2) Worst-case optimal join algorithms, that avoid redundant computations for queries with cycles; (3) Factorized representations, which compress intermediate and final query results; and (4) Advanced techniques for processing recursive queries, essential for traversing graph structures. For each component, we delve into its theoretical underpinnings, explore design considerations, and discuss the implementation challenges associated with integrating these techniques into existing database management systems. This survey aims to serve as a comprehensive resource for both researchers pushing the boundaries of query processing and practitioners seeking to implement state-of-the-art techniques, in addition to offering insights into future research directions in this rapidly evolving field.
Visualization recommendation systems help data analysts navigate large, complex datasets by generating visualizations of meaningful patterns, outliers, and insights that could influence downstream decision-making. However, recommendations can easily mislead or confuse analysts when they are not developed with care. In this survey, we review how visualization recommendation systems have been designed over the last 25 years and classify them by their underlying recommendation goals and high-level implementation strategies, including the user interfaces provided for navigating and interpreting the recommended visualizations. To understand their efficacy, we also review how visualization recommendation systems are evaluated in the literature. Given these observations, we present several open challenges and promising directions for future work in designing effective visualization recommendation systems.
