Reinforcement Learning Enhanced Zeroth-Order Optimization for Large-Scale Multiobjective Optimization Problems

“The curse of dimensionality” poses a major challenge to the scalability of multiobjective evolutionary algorithms (MOEAs) for large-scale multiobjective optimization problems (LSMOPs). Although existing MOEAs have achieved encouraging performance on problems with hundreds of decision variables, solving higher-dimensional LSMOPs remains computationally demanding. To mitigate these scalability challenges, we propose MOZO, an evolutionary zeroth-order (ZO) optimization method that combines zeroth-order directional estimation with a population-based MOEA backbone. Specifically, MOZO first estimates and aggregates objective-wise ZO gradients to provide search directions in high-dimensional decision spaces. It then refines these directions using acceleration and adaptation modules and embeds them within a population-based evolutionary search process to improve robustness against noisy estimates and objective conflicts. In addition, MOZO incorporates a reinforcement learning mechanism to adaptively configure suitable ZO and MOEA modules during the search process. Experimental results on benchmark LSMOPs with up to 100,000 decision variables demonstrate the effectiveness, efficiency, and scalability of MOZO compared with five state-of-the-art large-scale MOEAs. We further apply MOZO to the fine-tuning of OPT-2.7B and OPT-6.7B large language models on a single GPU, where the results show its practical potential under memory-constrained settings.