PD-DARTS - Progressive Discretization Differentiable Architecture Search.

Architecture design is a crucial step for neural-network-based methods, and it requires years of experience and extensive work. Encouragingly, with recently proposed neural architecture search (NAS), the architecture design process could be automated. In particular, differentiable architecture search (DARTS) reduces the time cost of search to a couple of GPU days. However, due to the inconsistency between the architecture search and evaluation of DARTS, its performance has yet to be improved. We propose two strategies to narrow the search/evaluation gap: firstly, rectify the operation with the highest confidence; secondly, prune the operation with the lowest confidence iteratively. Experiments show that our method achieves 2.46%/2.48% (test error, Strategy 1 or 2) on CIFAR-10 and 16.48%/16.15% (test error, Strategy 1 or 2) on CIFAR-100 at a low cost of 11 or 8 (Strategy 1 or 2) GPU hours, and outperforms state-of-the-art algorithms.