Diffusion Evolutionary Algorithm for Neural Architecture Search

Traditional Neural Architecture Search (NAS) methods often struggle with hyperparameter tuning and are prone to local optima, which limits the diversity of the population and hampers the search for optimal architectures. This paper introduces DiffEvo-NAS, a novel approach that leverages a diffusion evolution algorithm, which interprets the evolution process as a denoising operation and the reverse evolution as diffusion. By incorporating directional denoising (similar to directional selection) and adding small noise (analogous to mutations), the algorithm continuously refines individuals while maintaining diversity, enabling effective exploration of the search space and avoiding local optima. The effectiveness of DiffEvo-NAS is demonstrated through experiments on the CIFAR-10 and CIFAR-100 datasets, where it achieves a test error rate of 2.55% and 16.15%, respectively. On ImageNet, it achieves competitive performance with a top-1 error rate of 24.61% and top-5 error rate of 7.44%. These results confirm that DiffEvo-NAS offers significant improvements in both exploration and exploitation, outperforming other evolutionary NAS methods in terms of population diversity and architecture performance. As a stepping stone in the field, DiffEvo-NAS lays the groundwork for further innovations in NAS, particularly in the integration of diffusion models and evolutionary computation.