Multi-Modal Cross Learning for Improved People Counting using Short-Range FMCW Radar

Radar systems enable remote-less sensing of multiple persons in its field of view. In this paper, we propose a novel people counting system using 60-GHz frequency modulated continuous wave radar sensor. The proposed deep convolutional neural network learns from supervised radar data and also through knowledge distillation via multi-modal cross-learning of representation from a synchronized camera-based deep convolutional neural network. To overcome several shortcomings of the radar data, novel multi-modal cross learning algorithm is proposed that leverage the high-level abstractions learnt from camera modality. We also propose novel focal-regularized loss function to facilitate improved feature learning. We demonstrate the superior performance of our proposed solution in counting upto 4 people and detection of more than 4 people in indoor environment in comparison to the state-of-art radar-based uni-modal learning.