Learned indexes, while promising alternatives to traditional indexes like $\mathbf{B}^{+}$ tree, are hindered by expensive training costs and the need for high model accuracy. Existing optimization methods are often restrictive in scope, rely on specialized knowledge, and are challenging to deploy. We introduce the Learned Index Optimization Framework (LIOF), a general and easily integrated solution to expedite training and improve accuracy for both one- and multi-dimensional learned indexes. LIOF directly generates optimized initial parameters for index node models based on their local key distribution. Inspired by meta-learning, LIOF uses a novel data-driven encoder and a parameter-centric decoder to adaptively translate key distributions into optimized model initializations. Extensive experiments on real-world and synthetic datasets demonstrate that LIOF provides substantial enhancements in both training efficiency and predictive accuracy, reducing build times by up to 69% and query costs by up to 67%.

Cardinality and cost estimation are critical components of query optimization, as they directly influence the construction of efficient physical execution plans. While machine learning-based estimators have achieved notable success, they face several challenges: (1) Training data derived from rigid, template-driven benchmarks exhibits significant distributional divergence from real-world query workloads, a challenge further compounded by the manual template design in exhaustively representing the full spectrum of query patterns. (2) These methods demonstrate limited generalization capabilities, especially in scenarios involving sub-plan estimation or queries that significantly deviate from the training query templates. Furthermore, the inherent inefficiency of operator-level cardinality estimation frequently undermines its applicability for accurate cost estimation. (3) These approaches frequently fail to leverage the rich semantic information and dynamic dependencies between operators. To address these challenges, we propose a novel operator-level cardinality and cost estimator that simultaneously estimates the cardinality and cost of all sub-plans within a query plan. First, we leverage large language models to generate high-quality and diverse SQL queries, which serve as the foundation for pre-training and fine-tuning our model. Second, we introduce a semantic-based operator encoding strategy, augmented with a novel tree-structure-aware neural network, to effectively represent each sub-plan. Third, we propose a specialized loss function tailored for joint cardinality and cost prediction at the operator level, fully utilizing labels from each sub-plan. Extensive experiments on both synthetic and real-world datasets demonstrate that our method consistently outperforms state-of-the-art approaches.

Query optimizer is a critical component of a database management system, significantly impacting both user experience and resource utilization. To further enhance the capabilities of query optimizers, recent research has explored the application of deep learning. They either learn to construct plans from scratch in a bottom-up manner or steer the plan generation behavior of the traditional optimizer using hints. While they have shown promising results on certain workloads, they often suffer from low training efficiency or a constrained plan search space. To address these challenges, we introduce FOSS, a novel learned framework for query optimization. Our key insight is that while traditional optimizers may produce suboptimal plans due to estimation errors, only minor adjustments to specific suboptimal nodes are sufficient to achieve a highly efficient execution plan. FOSS assigns a reinforcement learning-based planner to refine the original plan generated by the traditional optimizer, making targeted modifications through a sequence of actions. Additionally, an evaluator is employed to assess candidate plans and select the final execution plan. To accelerate planner training, we integrate the evaluator and the traditional optimizer to construct a simulated environment, which enables the rapid generation of high-quality simulated experiences for planner updates. We evaluate the performance of FOSS on PostgreSQL and MySQL using the Join Order Benchmark, TPC-DS, and Stack Overflow workloads. The experimental results demonstrate that FOSS outperforms the state-of-the-art methods in terms of latency performance. FOSS achieves a total latency speedup of 1.30x to 9.09x over PostgreSQL and 1.35x to 7.14x over MySQL across various benchmarks.

Learned indexes, emerging as a promising alternative to traditional indexes like B+Tree, utilize machine learning models to enhance query performance and reduce memory usage. However, the widespread adoption of learned indexes is limited by their expensive training cost and the need for high accuracy of internal models. Although some studies attempt to optimize the building process of these learned indexes, existing methods are restrictive in scope and applicability. They are usually tailored to specific index types and heavily rely on pre-trained model knowledge, making deployment a challenging task. In this work, we introduce the Learned Index Optimization Framework (LIOF), a general and easily integrated solution aimed at expediting the training process and improving the accuracy of index model for one-dimensional and multi-dimensional learned indexes. The optimization of LIOF for the learned indexes is intuitive, directly providing optimized parameters for index models based on the distribution of node data. By leveraging the correlation between key distribution and node model parameters, LIOF significantly reduces the training epochs required for each node model. Initially, we introduce an optimization strategy inspired by optimization-based meta-learning to train the LIOF to generate optimized initial parameters for index node models. Subsequently, we present a data-driven encoder and a parameter-centric decoder network, which adaptively translate key distribution into a latent variable representation and decode it into optimized node model initialization. Additionally, to further utilize characteristics of key distribution, we propose a monotonic regularizer and focal loss, guiding LIOF training towards efficiency and precision. Through extensive experimentation on real-world and synthetic datasets, we demonstrate that LIOF provides substantial enhancements in both training efficiency and the predictive accuracy for learned indexes.

Various works have utilized deep learning to address the query optimization problem in database system. They either learn to construct plans from scratch in a bottom-up manner or steer the plan generation behavior of traditional optimizer using hints. While these methods have achieved some success, they face challenges in either low training efficiency or limited plan search space. To address these challenges, we introduce FOSS, a novel framework for query optimization based on deep reinforcement learning. FOSS initiates optimization from the original plan generated by a traditional optimizer and incrementally refines suboptimal nodes of the plan through a sequence of actions. Additionally, we devise an asymmetric advantage model to evaluate the advantage between two plans. We integrate it with a traditional optimizer to form a simulated environment. Leveraging this simulated environment, FOSS can bootstrap itself to rapidly generate a large amount of high-quality simulated experiences. FOSS then learns from these experiences to improve its optimization capability. We evaluate the performance of FOSS on Join Order Benchmark, TPC-DS, and Stack Overflow. The experimental results demonstrate that FOSS outperforms the state-of-the-art methods in terms of latency performance. Compared to PostgreSQL, FOSS achieves speedup ranging from 1.15x to 8.33x in total latency across different benchmarks.

Knob tuning plays a crucial role in optimizing databases by adjusting knobs to enhance database performance. However, traditional tuning methods often follow a Try-Collect-Adjust approach, proving inefficient and database-specific. Moreover, these methods are often opaque, making it challenging for DBAs to grasp the underlying decision-making process. The emergence of large language models (LLMs) like GPT-4 and Claude-3 has excelled in complex natural language tasks, yet their potential in database knob tuning remains largely unexplored. This study harnesses LLMs as experienced DBAs for knob-tuning tasks with carefully designed prompts. We identify three key subtasks in the tuning system: knob pruning, model initialization, and knob recommendation, proposing LLM-driven solutions to replace conventional methods for each subtask. We conduct extensive experiments to compare LLM-driven approaches against traditional methods across the subtasks to evaluate LLMs' efficacy in the knob tuning domain. Furthermore, we explore the adaptability of LLM-based solutions in diverse evaluation settings, encompassing new benchmarks, database engines, and hardware environments. Our findings reveal that LLMs not only match or surpass traditional methods but also exhibit notable interpretability by generating responses in a coherent “chain-of-thought” manner. We further observe that LLMs exhibit remarkable generalizability through simple adjustments in prompts, eliminating the necessity for additional training or extensive code modifications. Drawing insights from our experimental findings, we identify several opportunities for future research aimed at advancing the utilization of LLMs in the realm of database management.

随着大规模低轨卫星星座的全面部署，对天基测控服务模式提出了新的挑战。分析了低轨卫星星座在常态运行、应急使用以及智能化应用中的测控需求；构建了新型天基测控服务系统，设计了前返向实时跟踪波束和全域静态波束组合应用的天基测控资源保障模式。提出了基于地面运管中心统一调度的天基测控服务模式，给出了即时计划驱动和“返向全时分发+前向业务驱动”两种典型模式下的系统工作流程，归纳总结了所述服务模式产生的效益。详细梳理了与所述服务模式相适配的卫星终端设计要求，明确了不同模式下的卫星终端工作状态。可用于指导后续对低轨卫星星座天基测控服务的工程化应用。

The accuracy of selectivity estimation is of vital importance to create good query plans in database management systems. We propose MOSE, a learning-based MOnotonic Selectivity Estimator, to provide accurate, reliable, and efficient selectivity estimation for query optimization.

Query optimization is crucial for the query performance of database systems. Despite decades of efforts from both research and industrial communities, query optimization remains one of the most challenging problems. Thanks to the advances in artificial intelligence, data-driven and learning-based techniques are seeing traction in database research recently. However, most former learning-based works perform less practical because they are evasive about the interaction between learning components and database systems. In this demonstration, we introduce DeepO, a novel deep-learning-based query optimizer that offers high-quality and fine-grained query optimization efficiently and practically. We implement DeepO and incorporate it into PostgreSQL, and we also provide a web user interface, where users can carry out the optimization operations interactively and evaluate the optimization performance. Preliminary results show that DeepO outperforms the baseline PostgreSQL optimizer.

大数据时代,数据规模庞大、数据管理应用场景复杂,传统数据库和数据管理技术面临很大的挑战.人工智能技术因其强大的学习、推理、规划能力,为数据库系统提供了新的发展机遇.人工智能赋能的数据库系统通过对数据分布、查询负载、性能表现等特征进行建模和学习,自动地进行查询负载预测、数据库配置参数调优、数据分区、索引维护、查询优化、查询调度等,以不断提高数据库针对特定硬件、数据和负载的性能.同时,一些机器学习模型可以替代数据库系统中的部分组件,有效减少开销,如学习型索引结构等.分析了人工智能赋能的数据管理新技术的研究进展,总结了现有方法的问题和解决思路,并对未来研究方向进行了展望.