MT-DyNN: Multi-Teacher Distilled Dynamic Neural Network for Instance-Adaptive Detection in Autonomous Driving

Multi-object detection in autonomous driving faces challenges due to multi-scale entities, diverse streetscapes, and limited computational resources. To address these challenges, we present MT-DyNN, a Multi-Teacher knowledge-distilled Dynamic Neural Network framework for instance-adaptive detection, optimizing detection accuracy and inference cost in autonomous driving. The framework’s student network comprises a customizable multi-branch residual detection network and a lightweight policy network. The former efficiently extracts multi-scale features in parallel without altering receptive fields, while the latter, depending on curriculum learning, captures task-relevant features and dynamically generates routing vectors to guide the activation or deactivation of residual blocks according to image instance complexity. The framework’s teacher network employs a soft-voting strategy to consolidate knowledge from multiple pre-trained teacher models, providing consistent guidance to the student. Within this distillation paradigm, the policy network’s routing search space is gradually refined, and the policy and detection networks are jointly fine-tuned to optimize the alignment between routing decisions and feature extraction. Experimental results on CIFAR and ImageNet demonstrate that compared to early exiting and stochastic depth methods, MT-DyNN achieves higher accuracy at the same inference cost and reduces the cost by 50% and 59% at comparable accuracy levels. The generated routing maintains channel sparsity across diverse scenarios.