Progressive Neural Index Search for Database System

In database systems, index is the key component to support efficient search. Modern DBMS include many index implementations, and the DBA will choose a specific indexing strategy for the target application scenarios. However, new index structures are still being proposed to support new types of queries, which are tedious tasks involving plenty of research work. Even an experienced database researcher cannot tell which index is the best solution for a specific application. To address this issue, we propose a new approach, NIS (Neural Index Search), which searches for the optimal index structure using a neural network. The idea is analogy to the NAS (neural architecture search). We formalize the index structure as ordered and unordered blocks and apply a reinforcement learning model to organize the blocks into an index structure, optimized for a given workload and dataset. NIS can simulate many existing index structures. Experiments show that the auto-generated index by NIS can achieve a comparable performance with the state-of-the-art index.