A hybrid algorithm integrating a couple of individual evolutionary algorithms (sub-algorithms) is widely recognized as an effective approach to enhance both robustness and optimization performance. Nevertheless, such integration often destroys the structure of the sub-algorithm and makes it difficult to incorporate additional evolutionary algorithms. To address these limitations, this study introduces a novel framework, the Heterogeneous Alternating Evolutionary Algorithm (HAEA), designed to integrate multiple evolutionary algorithms while enabling the flexible addition, removal, and replacement of internal sub-algorithms. To facilitate the integration of a broad spectrum of sub-algorithms, this study draws inspiration from the particle swarm optimization algorithm to devise a suite of information indicators for the transmission of optimization information between sub-algorithms with disparate structures. Furthermore, HAEA is endowed with an adaptive mechanism that dynamically modifies the selection probabilities of its sub-algorithms based on their long-term and short-term performance throughout the evolutionary process. We conducted a comparative analysis of HAEA against all its sub-algorithms across three widely recognized function test sets: CEC2013, CEC2017, and CEC2022. Meanwhile, we applied the HAEA separately to basic metaheuristic algorithms and advanced evolutionary algorithms in recent years and conducted two comparative experiments. Both experimental results show that HAEA outperforms all sub-algorithms in terms of robustness and optimization performance. Its distinctive flexibility allows for the incorporation of additional superior evolutionary algorithms in the future, thereby enhancing its overall performance.