Joint Sparsity Aided Joint Manifold Learning for Sensor Fusion

This paper presents a joint sparsity aided joint manifold learning (JSAJML) approach for heterogenous sensor fusion of image and radio frequency (RF) data. A typical scenario includes several objects (with RF emitters), which are observed by a network of platforms with Medium Wavelength Infrared (MWIR) cameras and/or RF Doppler sensors. In the past, we developed two heterogeneous approaches for mixed and/or mixed sensor modalities. The joint sparsity (JS) approach for cases without ground truth and joint manifold learning (JML) for cases with ground truth. In this paper, we unify the abovementioned methods. We propose an interactive learning/training of JML based on joint sparsity results. We use the joint sparsity support recovery to generate estimated ground truth and train the JML framework. On the other hand, the joint sparsity can use the JML results to improve the data fusion results and speed up the processing as well. The JSAJML approach is tested and verified on AFRL datasets.