Fluid-DataTable: Elastic and Efficient Caching for Cloud Native Big Data Query System.

Nowadays, big data query systems are often deployed on cloud native platforms for advantages like automated deployment and elastic scalability. Nonetheless, traditional optimization approaches often fail to fully exploit the potential of cloud native platforms and cache acceleration. It results in cache related QoS drawbacks for running query on cloud native environment, including inadequate space for caching, static cache configurations that are unsuitable for dynamic workloads, and query task scheduling that lacks consideration for cache reuse. To address these issues, we propose Fluid-DataTable, an elastic and efficient table cache management and query task scheduling service designed specifically for query systems on cloud native platforms. It prioritizes loading tables with high acceleration gain into cache cluster, dynamically adjusts the number of cache replicas and cache nodes based on data access frequency, and plans the execution order of queries with consideration of the cache state. Experimental results show that Fluid-DataTable achieves significant improvement in cache efficiency and query performance due to the proposed cloud native cache management and adaptation mechanisms. Also, the cache-aware scheduling strategy reduces cache replacement frequency, bringing about around 30