Neural Architecture Search with Zero-Cost Proxies

Neural Architecture Search (NAS) has demonstrated great potential in automating the design of deep neural networks. However, computational complexity and resource requirements are major drawbacks for neural architecture search algorithms. Zero-cost proxies offer a solution by assessing architecture performance using inexpensive surrogate metrics. In this paper, two novel methods are introduced to integrate zero-cost proxies into evolutionary neural architecture search: proxy-guided crossover and proxy regularization. In the proxy-guided crossover approach, the cut-point crossover operation is guided by the zerocost proxy scores, in order to choose the best possible combination of the genetic material from two parents. Proxy regularization integrates zero-cost proxies as regularization during neural architecture search to eliminate the model with the lowest proxy score from the population list. The proposed methods were evaluated using CIFAR-100 and NAS-Bench-201 benchmark datasets. The experimental analysis demonstrates that the proposed techniques speed up the search process, helping in finding better models faster. This research contributes to the neural architecture search field by making the discovery of promising neural network architectures during the search process more effective.