A Matrix Factorization Based Framework for Fusion of Physical and Social Sensors

Our world is witnessing the on-going substantial increase in the number of multimodal physical and social sensors that are ubiquitously distributed and observing or reporting what is happening in their surroundings. These sensors provide massive amounts of spatio-temporal digital footprints which can be analyzed for various tasks such as event detection or situation awareness. However, inherent noise due to the nature of these sensors result in imprecise data and hence imprecise analysis. Also, the heterogeneous data from different modalities, formats and sources make interpreting different levels of information a big challenge. To overcome these limitations, we propose a novel unified matrix factorization-based model to fuse physical and social sensor signals for spatio-temporal analysis. Readings of physical sensor signals are represented by a spatio-temporal situation matrix, which then incorporates social content that can provide explanations for the signal strengths. We test our framework on large-scale real-world data including PSI stations data, traffic CCTV camera images, and tweets for situation prediction as well as for filtering noise to detect events of diverse situations. The experimental results suggest that the proposed matrix factorization approach can utilize the sources correlation, resulting in better performances in various situational understanding tasks.