Multi-Vision Services Acceleration Framework for IoT Devices

Deploying a single deep neural network (DNN) model on ubiquitous Internet of Things (IoT) devices to provide multiple vision services (e.g., semantic segmentation service, facial attribute recognition service) has attracted significant interest from industry and academia. However, the most relevant studies have two limitations: (i) the multiple services DNN commonly has low performance due to the gradient interference between different vision services; (ii) the multiple services DNN has low inference speed on IoT devices. To this end, this paper introduces a multiple vision services acceleration framework for IoT devices. The proposed MCBNet contains a novel multi-controllable branching structure to control different services explicitly for improving multiple services performance and a service balance layer with the per-channel trainable masks in specific branches for mitigating gradient interference. In deployment, the proposed framework converts the multi-branch topology MCBNet model into the plain topology Rep-MCBNet model by re-parameterization technique for accelerating services inference speed. Experimental results on the CelebA and Cityscapes datasets show that our proposed framework outperforms the state-of-the-art approaches in terms of performance and inference speed. In particular, we deploy the Rep-MCBNet on the laptop with Intel mobile processors demonstrating that the proposed framework achieves 1.9× faster processing frames per second than ETR-NLP, with 1.5× fewer parameters and 1.3× fewer floating-point operations.