Existing database knob tuning methods evaluate the actual performance of each configuration by fully executing the entire workload. However, our experimental analysis reveals that this exhaustive execution approach significantly limits tuning efficiency, particularly when dealing with underperforming configurations. To address this issue, we propose ESTune, which is designed to early-stop the execution of poorly performing configurations. ESTune approximates the actual performance of these configurations using high-confidence predicted values generated from partially executed workload data and configuration knob settings. This strategy significantly reduces the evaluation time for underperforming configurations while maintaining the overall tuning effectiveness. The high-confidence predicted values are produced by a Hybrid Bayesian Neural Network (HBNN), which models the performance distribution with respect to different knob configurations. To address the challenge of limited training data commonly encountered in database knob tuning, ESTune integrates a Model-Agnostic Meta-Learning (MAML), thereby enhancing the few-shot learning capability of the HBNN. Extensive evaluations on a wide range of workloads consistently demonstrate that ESTune improves the tuning efficiency of existing methods.
Personalized agents are increasingly applied to assist users across a wide range of tasks. Effective personalized assistance requires not only retrieving explicit facts from past interactions stored in agent memory, but also inferring abstract personal characteristics. However, existing memory benchmarks primarily evaluate whether an agent can retrieve information explicitly stated in conversational histories, failing to provide an effective assessment of deeper user understanding. In this work, we propose Setoka, a benchmark for evaluating memory-augmented personalized agents with hierarchical user understanding from heterogeneous data. Grounded in theories from cognitive and personality psychology, Setoka defines four levels of user understanding, i.e., semantic memory, episodic memory, behavior pattern, and personality trait. Moreover, to enable realistic yet privacy-preserving evaluation, we design a psychometrics-based pipeline that synthesizes diverse, coherent heterogeneous user data and queries at scale. Finally, we leverage Setoka to evaluate 3 language models combined with 5 memory systems for 10 synthetic users. Our comprehensive evaluation reveals that while existing systems perform well on semantic memory retrieval, their performance declines on episodic memory. Moreover, when dealing with behavior pattern and personality trait understanding tasks that require integrating heterogeneous and fragmented information dispersed over time, performance declines even further. These findings demonstrate that user understanding cannot be handled by simple fact retrieval, motivating the design of memory mechanisms for cross-source integration and abstraction over long-term user behavior.
Log-structured merge trees (LSM-trees) serve as the backbone of modern storage engines, yet their compaction process frequently becomes the primary performance bottleneck in production environments, compromising quality of service (QoS) in latency-sensitive applications. While LSM-tree based database systems expose numerous tunable parameters, existing tuning approaches focus exclusively on optimizing average-case throughput and latency, leaving compaction-induced performance bottlenecks unaddressed. In this work, we formulate robustness to compaction-induced performance variability as a novel constrained optimization problem, aiming to reduce performance variance without violating the throughput and latency service level agreements (SLAs). We propose MerTune, a knowledge-driven and constraint-aware Bayesian optimization framework designed for LSM-tree compaction tuning. MerTune models both the optimization objective and SLA constraints via Gaussian processes, while leveraging large language models (LLMs) to extract and group parameter knowledge from the database's official documentation. By integrating workload characterization, LLM-guided parameter selection, and knowledge-driven Bayesian exploration, MerTune improves robustness against compaction-induced performance fluctuations. MerTune has been applied to OceanBase database, and extensive evaluations under representative workloads demonstrate that it reduces throughput fluctuation by over 66.4%, latency fluctuation by over 45.5%, and CPU utilization fluctuation by over 17.1%, achieving superior optimization effectiveness and faster convergence compared to state-of-the-art tuning methods.
Large language models (LLMs) advance natural language (NL) interaction with databases by converting queries into SQL. However, users often lack familiarity with database schemas, making it difficult to express precise query requirements. To address this, we propose SQL-QMARS, a multi-agent Text-to-SQL framework designed to interactively clarify user intent. The system evaluates query vagueness using a three-layer metadata structure (theme, table, and field). Based on this evaluation, it dynamically triggers two flows: recommending multi-granular suggestions for vague queries and resolving ambiguities for clear ones. Furthermore, the system supports fusing external data to expand the knowledge source for query suggestions. The demonstration indicates that SQL-QMARS effectively guides users from vague to precise queries, improving the practicality of NL-based database interaction.
The stability and efficiency of database systems are critical to numerous data-intensive applications. However, it remains challenging to keep stable and high query performance, particularly under complex analytical workloads where subtle performance anomalies would cause significant latency and resource inefficiencies. Existing approaches often separate diagnosis from correction—focusing either on detecting execution anomalies or on black-box tuning techniques with limited interpretability and generality. In this paper, we propose Q-Doctor, a Query-level retrieval-augmented framework for Diagnosing and correcting database performance. First, Q-Doctor jointly encodes both query semantics and execution behaviors via a hybrid representation combining graph and tree neural encoders. This representation enables efficient retrieval of similar historical cases, which then guides an informed and fine-grained diagnosis. Moreover, a multi-agent correction module is introduced to collaboratively refine SQL hints and system configurations via reinforcement-guided iterations. Extensive experiments on well-established benchmarks demonstrate that Q-Doctor could accurately identify hidden performance anomalies and significantly improve query performance.
The integration of Large Language Models (LLMs) into database systems through SQL has made the data analytics workflow elegant. However, this integration introduces significant computational overhead, as conventional LLM inference is not optimized for relational workloads. Existing optimization techniques are often designed for unpredictable online request streams and treat the LLM engine as a black box, failing to leverage the prior knowledge of the entire data batch available in database operations. To bridge this gap, we propose BOND, a co-designed framework that deeply integrates batch-aware optimizations on both the inference engine and data organization sides. Specifically, BOND consists of two main components. First, a batch-efficient LLM inference engine indexes incoming prompts using a radix tree to identify shared prefix. Then, it models batch creation as a bin-packing problem, grouping computations to align with GPU hardware characteristics. As a result, it effectively mitigates tile-quantization effects. Second, a prefix-oriented data re-organizer optimizes the relational data before inference. This process uses a tree-based framework to find a data layout that maximizes prefix sharing and cache localities. Finally, these two components are bridged by a bubble-free task scheduler that ensures continuous GPU utilization in distributed environments. Experimental results show that our approach could reduce query execution time by up to 71.8%.
Integrating LLMs for data processing enables semantic querying but causes GPU memory bottlenecks and redundant computations. We present SemInfer, an acceleration system for batch semantic processing that treats the KV Cache as a semantic index, offloading pre-computed caches to host storage to eliminate redundancy. To reduce the index size, we propose a pruning strategy based on last-layer aggregated attention to accurately retain critical semantic tokens. Furthermore, we employ a pipeline mechanism to enable the asynchronous overlapping of CPU-GPU transmission and inference computation. This demonstration showcases the complete workflow of SemInfer on the IMDB dataset, achieving up to a 16.3x inference speedup over direct LLM inference and a 90
The SQL-based exploratory data analysis has garnered significant attention within the data analysis community. The emergence of large language models (LLMs) has facilitated the paradigm shift from manual to automated data exploration. However, existing methods generally lack the ability for cross-domain analysis, and the exploration of LLMs capabilities remains insufficient. This paper presents TiInsight, an SQL-based automated cross-domain exploratory data analysis system. First, TiInsight offers a user-friendly GUI enabling users to explore data using natural language queries. Second, TiInsight offers a robust cross-domain exploratory data analysis pipeline: hierarchical data context (i.e., HDC) generation, question clarification and decomposition, text-to-SQL (i.e., TiSQL), and data visualization (i.e., TiChart). Third, we have implemented and deployed TiInsight in the production environment of PingCAP and demonstrated its capabilities using representative datasets. The demo video is available at https://youtu.be/JzYFyYd-emI.
Practical implementations of Isolation levels (ILs) might deviate from their theoretical definitions, resulting in isolation bugs. Triggering isolation bugs usually requires concurrent test cases, which causes high debugging complexity. These test cases typically contain numerous irrelevant operations, resulting in large and redundant raw cases that violate the three essential criteria for bug report submission, i.e., reproducibility, conciseness , and uniqueness. Achieving these criteria involves three key challenges: 1) deterministically reproduce bugs despite high concurrency; 2) efficiently reduce cases without losing bug-triggering operations; 3) accurately distinguish unique bugs from the bug reports having complex transaction behaviors. To address these challenges, we propose an isolation bug case reduction and deduplication framework Pisco. First, we propose to simulate the DBMS's internal state to infer the order of conflicting operations for deterministic bug reproduction. Second, we introduce a dependency-aware divide-and-conquer strategy for efficient case reduction. Finally, we design a domain knowledge-driven, multi-agent collaboration framework for accurate bug deduplication. Extensive experiments show that Pisco reduces the cases to their minimal forms in just 20.0%/33.3% of the time required by C-Reduce / DDMin and has a deduplication ratio of up to 91.6%.
From professional research to everyday planning, many tasks are bottlenecked by wide-scale information seeking, which is more repetitive than cognitively complex. With the rapid development of Large Language Models (LLMs), automated search agents powered by LLMs offer a promising solution to liberate humans from this tedious work. However, the capability of these agents to perform such "wide-context" collection reliably and completely remains largely unevaluated due to a lack of suitable benchmarks. To bridge this gap, we introduce WideSearch, a new benchmark engineered to evaluate agent reliability on these large-scale collection tasks. The benchmark features 200 manually curated questions (100 in English, 100 in Chinese) from over 15 diverse domains, grounded in real user queries. Each task requires agents to collect large-scale atomic information, which could be verified one by one objectively, and arrange it into a well-organized output. A rigorous five-stage quality control pipeline ensures the difficulty, completeness, and verifiability of the dataset. We benchmark over 10 state-of-the-art agentic search systems, including single-agent, multi-agent frameworks, and end-to-end commercial systems. Most systems achieve overall success rates near 0\%, with the best performer reaching just 7\%. However, given sufficient time, cross-validation by multiple human testers can achieve a near 100\% success rate. These results demonstrate that present search agents have critical deficiencies in large-scale information seeking, underscoring urgent areas for future research and development in agentic search.
Isolation Levels (IL) act as correct contracts between applications and database management systems (DBMSs). The complex code logic and concurrent interactions among transactions make it hard to expose violations of various ILs stated by DBMSs. With the recent proliferation of new DBMSs, especially the cloud ones, there is an urgent demand for a general way to detect bugs violating various ILs. The core challenges come from the requirements of: (a) lightweight (verifying without modifying the application logic in workloads and the source code of DBMSs), (b) generality (verifying various ILs), and (c) efficiency (performing efficient verification on a long running workload). To this end, we propose a powerful and practical bug-finding tool Leopard. For lightweight, we propose to infer transaction dependencies based on the time intervals of operations collected from the client-side, without modifying the source code of DBMSs. For generality, based on a thorough analysis of existing concurrency control protocols, we summarize and abstract four mechanisms which can implement ILs in all commercial DBMSs we have investigated. For efficiency, we design a two-level pipeline to organize and sort massive time intervals in a time and memory conservative way; we propose a mechanism-mirrored verification to simulate the concurrency control protocols implemented in DBMSs for high throughputs; From experimental results, Leopard outperforms existing methods Cobra and Elle. In practice, Leopard has a superpower to verify various ILs on any workload running on all commercial DBMSs. Moreover, it has discovered 49 bugs undetected by other existing methods.
Distributed databases are widely used in various fields, such as financial services and e-commerce. These businesses generally exhibit characteristics of large-scale and rapid growth. However, these business systems often suffer from deadlocks that prevent them from operating normally for extended periods. Traditional deadlock detection methods face challenges in scalability and efficiency, especially as the number of nodes increases. Therefore, deadlock detection has always been a research area in distributed databases. In this paper, we introduce an efficient deadlock detection algorithm called HAWK, leveraging a Hierarchical Approach based on WorKload modeling. Our algorithm addresses these issues by constructing a dynamic hierarchical detection tree that adapts to transaction patterns, significantly reducing time complexity and communication overhead. HAWK first models the workload and generates a predicted access graph (PAG), transforming the problem of partitioning detection task in the basic hierarchical detection into partition detection zone (DZ) in the PAG by a graph-cutting algorithm. Then, leveraging the properties of strongly connected components (SCCs) and deadlock cycles, the SCC-cut algorithm naturally partitions the system-wide deadlock detection into multiple non-intersecting detection zones, thereby enhancing detection efficiency. We used the greedy SCC-cut algorithm to perform a more fine-grained partitioning of the complex PAG. Finally, by periodically sampling and updating the hierarchical structure, the algorithm remains responsive to dynamic workload variations, ensuring efficient detection. Our approach outperforms both centralized and distributed methods, offering a more efficient and adaptive solution. Extensive experimental results demonstrate the effectiveness of the HAWK algorithm, showing significant reductions in the duration of the deadlock and improved system throughput.
This study proposes a novel storage engine, SynchroStore, designed to address the inefficiency of update operations in columnar storage systems based on Log-Structured Merge Trees (LSM-Trees) under hybrid workload scenarios. While columnar storage formats demonstrate significant query performance advantages when handling large-scale datasets, traditional columnar storage systems face challenges such as high update complexity and poor real-time performance in data-intensive applications. SynchroStore introduces an incremental row storage mechanism and a fine-grained row-to-column transformation and compaction strategy, effectively balancing data update efficiency and query performance. The storage system employs an in-memory row storage structure to support efficient update operations, and the data is converted to a columnar format after freezing to support high-performance read operations. The core innovations of SynchroStore are reflected in the following aspects:(1) the organic combination of incremental row storage and columnar storage; (2) a fine-grained row-to-column transformation and compaction mechanism; (3) a cost-based scheduling strategy. These innovative features allow SynchroStore to leverage background computational resources for row-to-column transformation and compaction operations, while ensuring query performance is unaffected, thus effectively solving the update performance bottleneck of columnar storage under hybrid workloads. Experimental evaluation results show that, compared to existing columnar storage systems like DuckDB, SynchroStore exhibits significant advantages in update performance under hybrid workloads.
High-reliability distributed coordination services have become an indispensable part of modern large-scale distributed systems. Popular coordination services (e.g., ZooKeeper) adopt a single-writer design to provide a centralized service for managing system metadata, including various configuration information and data catalogs, and to provide distributed synchronization functions. With the continuous increase in metadata size and the scale of distributed systems, these coordination services gradually become performance bottlenecks due to their limitations in capacity, read and write performance, and scalability. To bridge the gaps, we propose FDBKeeper, a novel solution that enables scalable coordination services on distributed ACID key-value database systems. Our motivation is that transactional key-value stores (i.e., FoundationDB) meet the demands of performance and scalability required by large-scale distributed systems over coordination service. To leverage these advantages, coordination services can be implemented as an upper layer on top of distributed ACID key-value databases. Our experimental results demonstrate that FDBKeeper significantly outperforms ZooKeeper across key metrics. Additionally, FDBKeeper reduces hardware resource costs on average by 33% in the production environment, resulting in substantial monetary cost savings. We have successfully replaced ZooKeeper with FDBKeeper in the production-grade ClickHouse cluster deployment.