The DEEM'24 workshop (Data Management for End-to-End Machine Learning) is held on Sunday June 9th, in conjunction with SIGMOD/PODS 2024. DEEM brings together researchers and practitioners at the intersection of applied machine learning, data management and systems research, with the goal to discuss the arising data management issues in ML application scenarios. The workshop solicits regular research papers (8 pages) describing preliminary and ongoing research results, including industrial experience reports of end-to-end ML deployments, related to DEEM topics. In addition, DEEM 2023 has a category for short papers (4 pages) as a forum for sharing interesting use cases, problems, datasets, benchmarks, visionary ideas, system designs, preliminary results, and descriptions of system components and tools related to end-to-end ML pipelines. This year, the workshop received 16 high-quality submissions on diverse topics relevant to DEEM, of which 8 regular papers and 8 short papers.
SQL has attained widespread adoption, but Business Intelligence tools still use their own higher level languages based upon a multidimensional paradigm. Composable calculations are what is missing from SQL, and we propose a new kind of column, called a measure, that attaches a calculation to a table. Like regular tables, tables with measures are composable and closed when used in queries. SQL-with-measures has the power, conciseness and reusability of multidimensional languages but retains SQL semantics. Measure invocations can be expanded in place to simple, clear SQL. To define the evaluation semantics for measures, we introduce context-sensitive expressions (a way to evaluate multidimensional expressions that is consistent with existing SQL semantics), a concept called evaluation context, and several operations for setting and modifying the evaluation context.
Many big data systems are written in languages such as C, C++, Java, and Scala for high efficiency, whereas data analysts often use Python to conduct data wrangling, statistical analysis, and machine learning. User-defined functions (UDFs) are commonly used in these systems to bridge the gap between the two ecosystems. Debugging complex UDFs in data-processing systems is challenging due to the required coordination between language debuggers and the data-processing engine, as well as the debugging overhead on large volumes of data. In this paper, we showcase Udon, a novel debugger to support line-by-line debugging of UDFs in data-processing systems. Udon encapsulates modern line-by-line debugging primitives, such as those to set breakpoints, perform code inspections, and make code modifications while executing a UDF on a single tuple. In this demonstration, we use real-world scenarios to showcase the experience of using Udon for line-by-line debugging of a UDF.
In recent times, learned query optimizer has becoming a hot research topic in learned databases. It serves as the most suitable experimental plots for utilizing numerous machine-learning techniques and exhibits its superiority with enough evidence. In this tutorial, we aim to provide a wide and deep review and analysis on this field, ranging from theory to practice. At first, we would categorize and introduce representative methods for each learned component in the query optimizer, as well as for the end-to-end learned query optimizer. Then, we describe some benchmark evaluations and prototype applications. Their results have exhibited the bright future of applying learned query optimizers in practice. Based on them, we describe a cutting edge system with step-by-step guidelines. It is a middleware proposed recently to reduce the difficulties of developing and deploying learned algorithms in databases. It would help researchers to iterate their work and make learned query optimizers truly applicable in production. Finally, we summarize and point out several future directions. We hope this tutorial could inspire and guide both researchers and engineers working on learned query optimizers, as well as other contexts in learned databases.
We demonstrate the Approximate Selection Query Processing (ASQP-RL) system, which uses Reinforcement Learning to select a subset of a large external dataset to process locally in a notebook during data exploration. Given a query workload over an external database and notebook memory size, the system translates the workload to select-project-join (non-aggregate) queries and finds a subset of each relation such that the data subset - called the approximation set - fits into the notebook memory and maximizes query result quality. The data subset can then be loaded into the notebook, and rapidly queried by the analyst. Our demonstration shows how ASQP-RL can be used during data exploration and achieve comparable results to external queries over the large dataset at significantly reduced query times. It also shows how ASQP-RL can be used for aggregation queries, achieving surprisingly good results compared to state-of-the-art techniques.
Byzantine Fault-Tolerant (BFT) protocols have recently been extensively used by distributed and decentralized data management systems with non-trustworthy infrastructures to establish consensus on the order of transactions. BFT protocols cover a broad spectrum of design dimensions from infrastructure settings, such as the communication topology, to more technical features, such as commitment strategy and even fundamental social choice properties like order-fairness. The proliferation of different protocols has made it difficult to navigate the BFT landscape, let alone determine the protocol that best meets application needs. In this tutorial, we discuss BFT protocols that are used in modern large-scale data management systems, present a design space consisting of a set of design dimensions and explore several design choices that capture the trade-offs between different design space dimensions. The presented design space and its design choices will help developers analyze BFT protocols, understand how different protocols are related to each other, and find the protocol that best fits their needs.
Existing data debugging tools allow users to trace model performance problems all the way to the data by efficiently identifying slices (conjunctions of features and values) for which a trained model performs significantly worse than the entire dataset. To ensure accurate and fair models, one solution is to acquire enough data for these slices. In addition to crowdsourcing, recent data acquisition techniques design cost-effective algorithms to obtain such data from a union of external sources such as data lakes and data markets. We demonstrate PLUTUS, a tool for human-in-the-loop and model-aware data acquisition pipeline, on SystemDS, as an open source ML system for the end-to-end data science lifecycle. In PLUTUS, a user can efficiently identify problematic slices, connect to external data sources, and acquire the right amount of data for these slices in a cost-effective manner.
The recent explosion in the number and size of spatial remote sensing datasets from satellite missions creates new opportunities for data-driven approaches in domains such as climate change monitoring and disaster management. These approaches typically involve a feature engineering step that summarizes remote sensing pixel data located within zones of interest defined by another spatial dataset, an operation called zonal statistics. Although several spatial systems support zonal statistics operations, they differ significantly in terms of interfaces, architectures, and algorithms, making it hard for users to select the best system for a specific workload. To address this limitation, we propose Raven, a zonal statistics framework that provides users with a unified interface across multiple execution backends, while facilitating easy benchmarking and comparisons across systems. This demonstration showcases Raven 's multi-backend execution environment, domain-specific declarative language, optimization techniques, and benchmarking capabilities.
User-generated content like comments are valuable sources for various downstream applications. However, access to user comments data is often limited to specific platforms or outlets, which imposes a great limitation on the available data, and may not provide a representative sample of opinions from a diverse population on a particular event. This paper presents a comment crawling system that leverages the Web API of popular third-party commenting systems to collect comments from a large number of websites integrated with the commenting systems. Given a target page, the crawling system utilizes a deep learning model to extract API parameters and send HTTP requests to the API to retrieve comments. The system, Comquest, that we propose to demo is news-oriented and crawls comments regarding specific news topics/stories. Comquest can work with any website that allows commenting. Comquest provides a useful tool for collecting comments that represent a wider range of opinions, stances, and sentiments from websites on a global scale.
In the last two decades, we have been witnessing high demand for graph-based technologies in industry. On the research side, several recent advances have been made about large-scale graph processing, graph analytical systems and graph databases. The landscape of graph query languages is currently evolving with the definition of new standards, and the need for domain-specific languages to express graph algorithmic and analytical primitives will continue and increase in the next future. In this SIGMOD panel, we will discuss the impact of the above changes on the future of graph analytics. Is there a demand for more expressive languages and libraries for analyzing relationships in a graph? Are new hybrid OLTP/OLAP architectures required with improved performance and scalability? What are the graph analytical workloads and benchmarks that users expect on real-world graph applications? What will be the impact of graph ML on graph analytical systems? How to adapt these systems to the dynamic changes that are ubiquitous for graph-shaped data? These and other questions will be addressed in the panel.
Current knowledge graphs encompass diverse data formats, including images, text, tables, audio files, and videos. Additionally, the graph database ecosystem is required to support multiple co-existing data models. Addressing these challenges is essential for promoting interoperability between data sources. This demo introduces MillenniumDB, a high-performing, open-source graph database handling this diversity of data formats and models. MillenniumDB is a multi-modal, multi-model graph database, supporting the popular property graph paradigm, the Semantic Web format RDF, and the multi-layered graph model, which combines and extends the two. In terms of querying, its provides support for a Cypher-like language over property graphs and multilayered graphs, as well as SPARQL 1.1 support over RDF. The engine is build on a solid theoretical foundation and it leverages worst-case optimal join algorithms in combination with traditional relational query optimization. It also support a wide array of graph-specific tasks such as path finding, pattern recognition, and similarity search on multi-modal data. In this demo, we will showcase how MillenniumDB is currently being used to host three public multi-modal knowledge graphs. The first one, a multi-layered graph called TelarKG, was developed at IMFD Chile to track the information about the Chilean constitutional reform. In the second one, called BibKG, we integrate information about Computer Science publications from different sources and make them available as a property graph. Finally, for RDF, we provide a SPARQL endpoint for Wikidata, the largest knowledge graph openly available online. We remark that our endpoints have stable links, allowing the audience to post queries using their Web browser with no restrictions, and will be available during the review process and during the demo.
We demonstrate λ-Tune, a tool that leverages Large Language Models (LLMs) for automated, workload-adaptive database system tuning. λ-Tune harnesses the ability of LLMs to process and comprehend arbitrary textual data in a zero-shot manner, employing a workload-adaptive optimization approach. Given a database system, the hardware specifications, and a set of queries, λ-Tune generates prompts to retrieve configuration recommendations for the tuning knobs and the physical design, tailored to the specific system and workload. Our framework utilizes a workload compression approach that extracts and includes in the prompt only the most insightful workload characteristics, while the prompt size can be adjusted by a user-defined token budget. Utilizing the zero-shot capabilities of LLMs, λ-Tune outperforms other LLM and machine learning-enhanced database tuning baselines, which rely on time-consuming tuning and training phases, as well as expensive hardware, such as GPUs. During demonstration, users will be able to experiment with λ-Tune to tune Postgres and MySQL, as well as explore and modify the configurations retrieved by the LLM in an interactive way through λ-Tune's user interface.
Algorithmic recourse, or providing recommendations to individuals who receive an unfavorable outcome from an algorithmic system on how they can take action and change that outcome, is an important tool for giving individuals agency against algorithmic decision systems. Unfortunately, research on algorithmic recourse faces a fundamental challenge: there are no publicly available datasets on algorithmic recourse. In this work, we begin to explore a solution to this challenge by creating an agent-based simulation called The Game of Recourse (an homage to Conway's Game of Life) to synthesize realistic algorithmic recourse data. We designed The Game of Recourse with a focus on reliability and fairness, two areas of critical importance in socio-technical systems. You can access the application at https://game-of-recourse.streamlit.app.
We demonstrate Ver1 [10], a data discovery system that identifies project-join views over large repositories of tables that do not contain join path information. Ver solves both the technical (scale and search) and human (semantic ambiguity, navigating a large number of results) problems of view discovery.
Relational database management systems (RDBMSs) employ query optimizers to search for execution plans deemed optimal for specific queries. Classical optimizers rely on inaccurate parameter estimates and assumptions that may not hold true in real-world scenarios. Consequently, suboptimal execution plans may be chosen, leading to poor query execution performance. Recent proposals of learned query optimizers that leverage Machine Learning suffer also from the selection of suboptimal plans. In order to fill this gap, we have created Robust Workload Optimization (RobOpt), a prototype tool that facilitates robust execution of a query workload in RDBMSs. It implements a novel technique that takes workload logs as input, generates training samples, and trains a risk-aware learned cost model. It optimizes risk-aware plan selection strategies to achieve a desired level of runtime performance and robustness. In addition, it analyzes a workload according to its training samples and determines an optimal plan selection strategy either at the workload or query level. Ultimately, it enables the robust execution of any workload by determining an optimal plan selection strategy per query. RobOpt can work on top of any RDBMS.
GRADES-NDA is the premier workshop series on graph data management and analytics that aims to bring together researchers from academia, industry, and governmental organizations. GRADES-NDA'24 is a forum for discussing recent advances in (large-scale) graph data management and analytics systems, as well as proposing and discussing novel methods and techniques for addressing domain-specific challenges. In 2024, GRADES-NDA is in its seventh edition.
Causal analysis is an essential lens for understanding complex system dynamics in domains as varied as medicine, economics and law. Computer systems are often similarly complex, but much of the information about them is only available in long, messy, semi-structured log files. This demo presents Sawmill, an open-source system that makes it possible to extract causal conclusions from log files. Sawmill employs methods drawn from the areas of data transformation, cleaning, and extraction in order to transform logs into a representation amenable to causal analysis. It gives log-derived variables human-understandable names and distills the information present in a log file around a user's chosen causal units (e.g. users or machines), generating appropriate aggregated variables for each causal unit. It then leverages original algorithms to efficiently use this representation for the novel process of Exploration-based Causal Discovery - the task of constructing a sufficient causal model of the system from available data. Users can engage with this process via an interactive interface, ultimately making causal inference possible using off-the-shelf tools. SIGMOD'24 participants will be able to use Sawmill to efficiently answer causal questions about logs. We will guide attendees through the process of quantifying the impact of parameter tuning on query latency using real-world PostgreSQL server logs, before letting them test Sawmill on additional logs with known causal effects but varying difficulty. A companion video for this submission is available online.
Recent works show great interest in designing Graph Neural Networks (GNNs) that scale to large graphs. While previous work focuses on designing advanced sampling techniques for existing GNNs, the design of non-parametric GNNs, an orthogonal direction for scalable performance, has aroused lots of concerns recently. For example, nearly all top solutions in the Open Graph Benchmark leaderboard are non-parametric GNNs. Despite their high predictive performance and scalability, non-parametric GNNs still face two limitations. First, due to the propagation of over-smoothed features, they suffer from severe performance degradation along with the propagation depth. More importantly, they only consider the graph structure and ignore the feature influence during the non-parametric propagation, leading to sub-optimal propagated features. To address these limitations, we present non-parametric attention (NPA), a plug-and-play module that is compatible with non-parametric GNNs, to get scalable and deep GNNs simultaneously. We have deployed NPA in Tencent with the Angel platform, and we further evaluate NPA on both real-world datasets and large-scale industrial datasets. Experimental results on seven homophilic graphs (including the industrial Tencent Video graph) and five heterophilic graphs demonstrate NPA enjoys high performance -- achieves large performance gain over existing non-parametric GNNs, deeper architecture -- improves non-parametric GNNs with large model depth, and high scalability -- can support large-scale graphs with low time costs. Notably, it achieves state-of-the-art performance on the large ogbn-papers100M dataset.
Interest in data systems education is increasing, especially with the rise in demand for well-trained and re-trained data scientists. The database and the computing education research communities have complementary perspectives and experiences to share with each other. The DataEd workshop is organized as a dedicated venue for these communities to come together to share findings, cross-pollinate perspectives and methods, and shed light on opportunities for mutual progress in data systems education. In the DataEd 2024 workshop, we will present and discuss data management systems education experiences and research via keynote, paper presentations, and panel discussions.
Emergent distributed environments going beyond cloud computing offer new kinds of processing for massive and heterogeneous data, but also pose new challenges for data management. These challenges include among others: load balancing, dynamic data allocation on computing nodes, fault tolerance, handling low quality data, data and processes replication, resource provisioning, buffer management, query processing and optimization on different types of hardware, transaction management, and minimization of power usage. The BiDEDE workshop addresses these and other challenges related to building post-cloud distributed environments for data processing.
