Genetic Structural NAS: A Neural Network Architecture Search with Flexible Slot Connections

Selecting an appropriate neural network architecture and hyperparameters to optimize performance for a given application is a difficult task. To overcome this challenge, Neural Architecture Search (NAS) and Hyperparameter Optimization (HPO) have been introduced. However, these techniques are often computationally-expensive and require significant amounts of execution time. To address this issue, we propose a semi-automated NAS approach that optimizes pre-existing architectural structures using genetic algorithms and eliminates unsuccessful combinations through shallow network training. The effectiveness of our technique was verified through an experiment that produces a family of AlexNet-like neural networks comprising 1296 models for image classification tasks. The computational study was conducted with five runs of a genetic algorithm, resulting in the deep models with mean loss of 0.7044 and accuracy of 0.7334, both with low standard deviations, outperforming the original AlexNet model with a significant margin.