Surrogate-Assisted Evolutionary Neural Architecture Search with Architecture Knowledge Transfer

Neural Architecture Search (NAS) has emerged as a promising approach to automating the discovery of optimal neural network architectures. However, the computational cost of evaluating candidate architectures through full training presents a significant barrier to efficient search. While surrogate models can accelerate this process by predicting network performance, they struggle to accurately model the vast architecture search space when working with limited training data. To mitigate this challenge, we propose a Multi-Task, Multi-Surrogate Assisted Evolutionary NAS framework (MT-MSAENAS) that combines multiple surrogate models to enhance search efficiency. To fully exploit the limited training data, MT-MSAENAS constructs both strong and weak surrogate predictors, a global model (strong) that captures overall search space patterns and a local model (weak) that specializes in promising regions. Based on the two surrogate model, MT-MSAENAS employs evolutionary multi-tasking optimization by treating the strong and weak model-assisted search as two related optimization tasks to facilitate knowledge transfer between these models and improve the search efficiency. Experiments on the NAS-Bench101 and NAS-Bench201 search spaces show that the proposed algorithm outperforms state-of-the-art methods in architecture search.