Genetic Algorithm-Based Approach for Optimizing Query Performance in Big Data Environments

In light of the abundance and heterogeneity of data stemming from diverse, highly scalable, and distributed sources, information systems encounter novel challenges. The colossal datasets, characterized by diverse types, expansive storage ca-pacities' and unprecedented communication speeds, necessitate addressing increasingly intricate queries. Query optimization emerges as a predominant challenge within the realm of big data, primarily driven by concerns related to performance and cost. Consequently, we introduce our approach to optimizing the performance of HiveQL query action plans through the utilization of genetic algorithms, which exhibit efficacy across various domains, particularly in the domain of combinatorial optimization. Genetic algorithms circumvent the elevated costs associated with optimization efforts and provide adaptability by operating independently of problem-specific knowledge. These attributes render them a viable solution for addressing the query optimization predicament. The outcomes of our study underscore significant enhancements in performance, affirming the effective-ness of the proposed genetic algorithm-based approach.