Fusion of Optical and Millimeter Wave SAR Sensing for Object Recognition in Indoor Environment

Synthetic aperture radar (SAR) sensing at the millimeter-wave (mmWave) spectrum has emerging applications in indoor environments such as high-resolution imaging and localization, material characterization, and object recognition. Although a high-resolution environment map is obtained with SAR at the mmWave spectrum, recognition of objects based on shape is trivial, especially in the case of 2D imaging, where limited shape information is available. Therefore, this paper proposes a method for indoor object recognition in mmWave SAR sensing based on fusion with optical and mmWave SAR image fusion. The recognition includes detection, localization, and classification. Besides, extraction of geometric properties of the objects is also addressed. In this paper, firstly, a 2D multi-object indoor environment mapping is implemented at the spectrum of 68-92 GHz with a frequency modulated continuous wave (FMCW) radar. It replicates a setup of drone-based sensing with side-looking SAR. Two measurement scenarios are considered which primarily differ concerning object positions. The SAR image is reconstructed using a time-domain backprojection algorithm including a calibration procedure for generating a correct imaging plane. Finally, based on the presented method of SAR image fusion with the optical image at visible spectrum obtained using a CMOS camera, object recognition is evaluated.