Data discovery and preparation remain persistent bottlenecks in the data management lifecycle, especially when user intent is vague, evolving, or difficult to operationalize. The Pneuma Project introduces Pneuma-Seeker, a system that helps users articulate and fulfill information needs through iterative interaction with a language model-powered platform. The system reifies the user's evolving information need as a relational data model and incrementally converges toward a usable document aligned with that intent. To achieve this, the system combines three architectural ideas: context specialization to reduce LLM burden across subtasks, a conductor-style planner to assemble dynamic execution plans, and a convergence mechanism based on shared state. The system integrates recent advances in retrieval-augmented generation (RAG), agentic frameworks, and structured data preparation to support semi-automatic, language-guided workflows. We evaluate the system through LLM-based user simulations and show that it helps surface latent intent, guide discovery, and produce fit-for-purpose documents. It also acts as an emergent documentation layer, capturing institutional knowledge and supporting organizational memory.

Data analysts working with relational data often start with vague or underspecified questions and refine them iteratively as they explore the data. To support this iterative process, we demonstrate Pneuma-Seeker, a system that reifies a user's information need as explicit, inspectable relational specifications, enabling iterative refinement of the information need, targeted data discovery, and provenance-aware execution. Through two real-world procurement use cases, we show how Pneuma-Seeker leverages LLMs as transparent, interactive analytical collaborators rather than opaque answer engines.

Data markets promise to unlock data value by matching data suppliers with ML consumers. However, market design involves addressing intricate challenges, including data pricing, fairness, and robustness. We propose a pragmatic data-augmented AutoML market that seamlessly integrates with existing cloud-based AutoML platforms, such as Google’s Vertex AI. Unlike standard AutoML solutions, our design automatically augments buyer-submitted training data with valuable external datasets, pricing the resulting models based on their measurable performance improvements rather than computational costs as the status quo. Our key innovation is a pricing mechanism grounded in the instrumental value—the marginal model quality improvement—of externally sourced data. This approach bypasses direct dataset pricing complexities and accommodates diverse buyer valuations through menu-based options, thus providing an economically sustainable framework for monetizing external data.

Data sharing is central to various applications such as fraud detection, ad matching, and improving patient care. However, each solution to data sharing is bespoke and cost-intensive, hampering value generation. We identify the lack of abstractions to control data release as the culprit of the problem. For example, it is common to have constraints on whether to share data that depend on the result of sharing, and evaluating these constraints requires sharing in the first place, leading to a standstill. To help people build solutions to a wide variety of data sharing applications, we propose programmable dataflows, which consist of two components. The first component is an abstraction, the contract, which agents use to communicate the intent of a data sharing action and evaluate its consequences before the dataflow takes place. This helps agents control the release of their data. The second component is a contract programming model (CPM), which allows agents to program data sharing applications catered to each problem’s needs with the contract abstraction. We describe how to deploy those applications on a data escrow to ensure data remains protected from unintended data releases. Our evaluation shows 1) the contract abstraction permits representing a wide range of sharing problems, 2) CPM permits writing programs for complex data sharing problems and 3) quantitatively, our improvements to CPM make sharing programs run efficiently.

The standard objection to data compensation is that assigning value to individual data points is computationally infeasible at the scale of modern training. We argue that this objection is weaker than it appears because the training process already produces useful valuation signals. In particular, optimized data mixtures reveal information about the relative value of broad data sources, while scaling laws provide a benchmark for the aggregate share of model value attributable to data. We formalize the mixture-weight argument in a simple statistical model with correlated observations within sources and independent observations across sources, showing that, under a homoskedastic benchmark, optimal mixture shares equal relative information-value shares. We then extend the argument to neural-network training and characterize the corrections needed when data sources have heterogeneous return curves. Finally, we use standard scaling-law estimates to derive an upper-end benchmark for the aggregate data share of AI value. The result is a practical framework for coarse data valuation: use scaling laws to size the data pie, use optimized mixtures to divide it across sources, and route payments through a collective-management institution capable of aggregating claims.

The central concern of structural constitutional law is the organization of governmental power, but power comes in many forms. This Article is about how the law of structure regulates decision-making about, and popular control over, an increasingly potent form of power: the power government obtains from data. The government has always relied on information to meet its objectives, but the digitization of information over the last half century has yielded a distinctive form of governmental power-one that is liquid, transferable, minable, dynamic, and vital to virtually all governmental activity today. But despite the significant literature on private-sector "data governance," public law scholarship about data has focused centrally on privacy rights and far less on the structural law of data-and, in particular, the forms of data governance our constitutional democracy requires as data rises in importance as a form of governmental power. This Article develops an original account of data's structural law-the processes, institutional arrangements, transparency rules, and control mechanisms that, we argue, create distinctive structural dynamics for data's acquisition and appropriation to public projects. Doing so requires us to reconsider how law treats the category of power to which data belongs. Data is what we call an instrument of power-the means (money, land, arms, and the like) that the government uses to accomplish its many ends. The Constitution, we argue, facilitates popular control over material forms of power like data through specific and distinctive strategies, ranging from defaults to accounting mechanisms. Assessing data's structural ecosystem against that backdrop allows us to both map the structural law of data and provide an initial diagnosis of its deficits. Drawing on our respective fields-law and computer science-we conclude by suggesting legal and technical pathways to asserting greater procedural, institutional, and popular control over the government's data. Indeed, we argue that data has distinctive structural possibilities because of the capacity to both channel and constrain data through technical design.

Finding relevant tables among databases, lakes, and repositories is the first step in extracting value from data. Such a task remains difficult because assessing whether a table is relevant to a problem does not always depend only on its content but also on the context, which is usually tribal knowledge known to the individual or team. While tools like data catalogs and academic data discovery systems target this problem, they rely on keyword search or more complex interfaces, limiting non-technical users' ability to find relevant data. The advent of large language models (LLMs) offers a unique opportunity for users to ask questions directly in natural language, making dataset discovery more intuitive, accessible, and efficient. In this paper, we introduce Pneuma, a retrieval-augmented generation (RAG) system designed to efficiently and effectively discover tabular data. Pneuma leverages large language models (LLMs) for both table representation and table retrieval. For table representation, Pneuma preserves schema and row-level information to ensure comprehensive data understanding. For table retrieval, Pneuma augments LLMs with traditional information retrieval techniques, such as full-text and vector search, harnessing the strengths of both to improve retrieval performance. To evaluate Pneuma, we generate comprehensive benchmarks that simulate table discovery workload on six real-world datasets including enterprise data, scientific databases, warehousing data, and open data. Our results demonstrate that Pneuma outperforms widely used table search systems (such as full-text search and state-of-the-art RAG systems) in accuracy and resource efficiency.

Realizing a shared responsibility between providers and consumers is critical to manage the sustainability of HPC. However, while cost may motivate efficiency improvements by infrastructure operators, broader progress is impeded by a lack of user incentives. We conduct a survey of HPC users that reveals fewer than 30 percent are aware of their energy consumption, and that energy efficiency is among users' lowest priority concerns. One explanation is that existing pricing models may encourage users to prioritize performance over energy efficiency. We propose two transparent multi-resource pricing schemes, Energy- and Carbon-Based Accounting, that seek to change this paradigm by incentivizing more efficient user behavior. These two schemes charge for computations based on their energy consumption or carbon footprint, respectively, rewarding users who leverage efficient hardware and software. We evaluate these two pricing schemes via simulation, in a prototype, and a user study.

An open forum to discuss and debate the future of database research in the context of industry, other research communities, and AI.

Data powers economies, shapes societies, and fuels decision-making, yet its value remains poorly understood. Despite its centrality, we lack a unified framework for defining, measuring, and reasoning about data’s worth. This article develops a theory and systematization of the value of data—explaining why, how, and when data generates value. We distinguish data from documents, separate objective value from subjective judgments, and identify key dimensions of data’s worth. Our framework reconciles disparate notions of information, knowledge, and utility, offering insights that validate known principles while uncovering new opportunities to extract value from data. More than a taxonomy, this work provides a conceptual foundation for integrating perspectives from computer science, economics, and beyond. The conceptual foundation clarifies data’s role in technology, markets, and governance, advancing our ability to systematically understand and harness its value.

As hypothesis generation becomes increasingly automated, a new bottleneck has emerged: hypothesis assessment. Modern systems can surface thousands of statistical relationships—correlations, trends, causal links—but offer little guidance on which ones are novel, non-trivial, or worthy of expert attention. In this work, we study the complementary problem to hypothesis generation: automatic hypothesis assessment. Specifically, we ask—given a large set of statistical relationships, can we automatically assess which ones are novel and worth further exploration? We focus on correlations as they are a common entry point in exploratory data analysis that often serve as the basis for forming deeper scientific or causal hypotheses. To support automatic assessment, we propose to leverage the vast knowledge encoded in LLMs' weights to derive a prior distribution over the correlation value of a variable pair. If an LLM's prior expects the correlation value observed, then such correlation is not surprising, and vice versa. We propose the Logit-based Calibrated Prior, an LLM-elicited correlation prior that transforms the model’s raw output logits into a calibrated, continuous predictive distribution over correlation values. We evaluate the prior on a benchmark of 2,096 real-world variable pairs and it achieves a sign accuracy of 78.8%, a mean absolute error of 0.26, and 95% credible interval coverage of 89.2% in predicting Pearson correlation coefficient. It also outperforms a fine-tuned RoBERTa classifier in binary correlation prediction and achieves higher precision@K in hypothesis ranking. We further show that the prior generalizes to correlations not seen during LLM pretraining, reflecting context-sensitive reasoning rather than memorization.

In this paper, we present our experience integrating Rust, a memory-safe systems programming language, into an introductory database systems course project. Our findings indicate that while Rust’s steep learning curve posed initial challenges, it significantly enhanced students’ understanding of memory safety and systems programming concepts. We also discuss the outcomes of the course, which has now been taught to over 500 students over five separate offerings. While student feedback has been overwhelmingly positive, we provide insights for educators considering Rust for similar systems-oriented elective CS courses.

Data profoundly shapes our economic, political, and social ecosystems, yet we have limited control over its influence. Unchecked dataflows among agents can distort or undermine these ecosystems. Analyzing dataflows helps us understand their use and misuse, revealing opportunities to harness their value. Beneficial dataflows, such as hospitals sharing patient data, enhance healthcare outcomes, while harmful dataflows, such as personal data sold to self-interested brokers, cause substantial damage. Equally critical are absent dataflows-such as banks or governments withholding data due to competition or mistrust-that lead to unrealized value. Despite their substantial impact, unified methods to manage dataflows effectively are lacking. Current legal (regulations), economic (incentives), and technical (privacy technologies) interventions are developed independently without a clear evaluation of their collective effectiveness.

Reading comprehension models answer questions posed in natural language when provided with a short passage of text. They present an opportunity to address a long-standing challenge in data management: the extraction of structured data from unstructured text. Consequently, several approaches are using these models to perform information extraction. However, these modern approaches leave an opportunity behind because they do not exploit the relational structure of the target extraction table. In this paper, we introduce a new model, Relation Coherence, that exploits knowledge of the relational structure to improve the extraction quality. We incorporate the Relation Coherence model as part of FabricQA-Extractor, an end-to-end system we built from scratch to conduct large scale extraction tasks over millions of documents. We demonstrate on two datasets with millions of passages that Relation Coherence boosts extraction performance and evaluate FabricQA-Extractor on large scale datasets.

Causal analysis is essential for gaining insights into complex real-world processes and making informed decisions. However, performing accurate causal analysis on observational data is generally infeasible, and therefore, domain experts start exploration with the identification of correlations. The increased availability of data from open government websites, organizations, and scientific studies presents an opportunity to harness observational datasets in assisting domain experts during this exploratory phase. In this work, we introduce NEXUS, a system designed to align large repositories of spatio-temporal datasets and identify correlations, facilitating the exploration of causal relationships. Nexus addresses the challenges of aligning tabular datasets across space and time, handling missing data, and identifying correlations deemed "interesting". Empirical evaluation on Chicago Open Data and United Nations datasets demonstrates the effectiveness of NEXUS in exposing interesting correlations, many of which have undergone extensive scrutiny by social scientists.

Differential privacy (DP) enables private data analysis but is difficult to use in practice. In a typical DP deployment, data controllers manage individuals' sensitive data and are responsible for answering data analysts' queries while protecting individuals' privacy; they do so by choosing $\epsilon$, the privacy loss budget, which controls how much noise to add to the query output. However, it is challenging for data controllers to choose $\epsilon$ because of the difficulty of interpreting the privacy implications of such a choice on the individuals they wish to protect. To address this challenge, we first derive a privacy risk indicator (PRI) directly from the definition of ex-post per-instance privacy loss in the DP literature. The PRI indicates the impact of choosing $\epsilon$ on individuals' privacy. We then leverage the PRI to design an algorithm to choose $\epsilon$ and release query output based on data controllers' privacy preferences. We design a modification of the algorithm that allows releasing both the query output and $\epsilon$ while satisfying differential privacy, and we propose a solution that bounds the total privacy loss when using the algorithm to answer multiple queries without requiring controllers to set the total privacy loss budget. We demonstrate our contributions through an IRB-approved user study and experimental evaluations that show the PRI is useful for helping controllers choose $\epsilon$ and our algorithms are efficient. Overall, our work contributes to making DP easier to use for controllers by lowering adoption barriers.

Data sharing is central to a wide variety of applications such as fraud detection, ad matching, and research. The lack of data sharing abstractions makes the solution to each data sharing problem bespoke and cost-intensive, hampering value generation. In this paper, we first introduce a data sharing model to represent every data sharing problem with a sequence of dataflows. From the model, we distill an abstraction, the contract, which agents use to communicate the intent of a dataflow and evaluate its consequences, before the dataflow takes place. This helps agents move towards a common sharing goal without violating any regulatory and privacy constraints. Then, we design and implement the contract programming model (CPM), which allows agents to program data sharing applications catered to each problem's needs. Contracts permit data sharing, but their interactive nature may introduce inefficiencies. To mitigate those inefficiencies, we extend the CPM so that it can save intermediate outputs of dataflows, and skip computation if a dataflow tries to access data that it does not have access to. In our evaluation, we show that 1) the contract abstraction is general enough to represent a wide range of sharing problems, 2) we can write programs for complex data sharing problems and exhibit qualitative improvements over other alternate technologies, and 3) quantitatively, our optimizations make sharing programs written with the CPM efficient.

Data is the new oil of the 21st century. The growing trend of trading data for greater welfare has led to the emergence of data markets. A data market is any mechanism whereby the exchange of data products including datasets and data derivatives takes place as a result of data buyers and data sellers being in contact with one another, either directly or through mediating agents. It serves as a coordinating mechanism by which several functions, including the pricing and the distribution of data as the most important ones, interact to make the value of data fully exploited and enhanced. In this article, we present a comprehensive survey of this important and emerging direction from the aspects of data search, data productization, data transaction, data pricing, revenue allocation as well as privacy, security, and trust issues. We also investigate the government policies and industry status of data markets across different countries and different domains. Finally, we identify the unresolved challenges and discuss possible future directions for the development of data markets.

As users migrate their analytical workloads to cloud databases, it is becoming just as important to reduce monetary costs as it is to optimize query runtime. In the cloud, a query is billed based on either its compute time or the amount of data it processes. We observe that analytical queries are either compute- or IO-bound and each query type executes cheaper in a different pricing model. We exploit this opportunity and propose methods to build cheaper execution plans across pricing models that complete within user-defined runtime constraints. We implement these methods and produce execution plans spanning multiple pricing models that reduce the monetary cost for workloads by as much as 56%. We reduce individual query costs by as much as 90%. The prices chosen by cloud vendors for cloud services also impact savings opportunities. To study this effect, we simulate our proposed methods with different cloud prices and observe that multi-cloud savings are robust to changes in cloud vendor prices. These results indicate the massive opportunity to save money by executing workloads across multiple pricing models.

Causal analysis is crucial for understanding cause-and-effect relationships in observed data to inform better decisions. However, conducting precise causal analysis on observational data is usually impractical, so domain experts often begin their exploration by identifying correlations. In this paper, we demonstrate Nexus, a system that aligns tabular datasets across space and time, handles missing data, and identifies correlations deemed "interesting", facilitating the exploration of causal relationships.