Building Optimal Neural Architectures Using Interpretable Knowledge

Neural Architecture Search is a costly practice. The fact that a search spacecan span a vast number of design choices with each architecture evaluationtaking nontrivial overhead makes it hard for an algorithm to sufficientlyexplore candidate networks. In this paper, we propose AutoBuild, a scheme whichlearns to align the latent embeddings of operations and architecture moduleswith the ground-truth performance of the architectures they appear in. By doingso, AutoBuild is capable of assigning interpretable importance scores toarchitecture modules, such as individual operation features and larger macrooperation sequences such that high-performance neural networks can beconstructed without any need for search. Through experiments performed onstate-of-the-art image classification, segmentation, and Stable Diffusionmodels, we show that by mining a relatively small set of evaluatedarchitectures, AutoBuild can learn to build high-quality architectures directlyor help to reduce search space to focus on relevant areas, finding betterarchitectures that outperform both the original labeled ones and ones found bysearch baselines. Code available athttps://github.com/Ascend-Research/AutoBuild