Hardware-Aware Iterative One-Shot Neural Architecture Search With Adaptable Knowledge Distillation for Efficient Edge Computing

The growing demand for edge applications calls for efficient and optimized deep neural network models. Neural Architecture Search (NAS) is instrumental in designing such models, but achieving optimal architectures quickly remains a key challenge. To address this, we propose Hardware-aware Iterative One-shot NAS (HIO-NAS), a highly efficient approach that iteratively explores architectures across predefined search spaces for depth, filter, and width, all while respecting hardware constraints. HIO-NAS operates in four main steps: full training, random search, hardware verification, and retraining with adaptable knowledge distillation, repeated for each search space. The hardware-aware mechanism incorporates a lookup table to evaluate and filter subnetworks sampled during random search, ensuring only the most promising candidates proceed to hardware verification. The top-performing subnetwork(s) are then deployed on the target hardware for further validation. The adaptable knowledge distillation technique dynamically adjusts the teacher model’s influence based on deviations in the cost function during training. By progressively refining search spaces, HIO-NAS reduces computational overhead and avoids convergence issues. Its hypernetwork design emphasizes chain-like structures, where layers maintain connectivity. Applied to MobileNet v2 on CIFAR-100 and YOLOv7 on BDD100K, HIO-NAS delivered significant improvements: a 2.1% accuracy gain and 20.8% reduction in latency for MobileNet v2, and better performance over YOLOv7 and its variants. Interestingly, the findings highlight that unit-sharing topologies excel in depth searches, whereas unit-non-sharing topologies perform better in filter and width searches. Overall, HIO-NAS showcases a robust capability for efficiently discovering high-performance, hardware-optimized architectures, making it ideal for edge applications.