FogSurv: A Fog-Assisted Architecture for Urban Surveillance Using Artificial Intelligence and Data Fusion

Urban surveillance
, of which airborne urban surveillance is a vital constituent, provides
situational awareness
(SA) and timely response to emergencies. The significance and scope of urban surveillance has increased manyfold in recent years due to the proliferation of unmanned aerial vehicles (UAVs), Internet of things (IoTs), and multitude of sensors. In this article, we propose FogSurv—a fog-assisted surveillance architecture and framework leveraging artificial intelligence (AI) and information/data fusion for enabling real-time SA and monitoring. We also propose an AI- and data-driven information fusion model for FogSurv to help provide (near) real-time SA, threat assessment, and automated decision-making. We further present a latency model for AI and information fusion processing in FogSurv. We then discuss several use cases of FogSurv that can have a huge impact on multifarious fronts of national significance ranging from safeguarding national security to monitoring of critical infrastructures. We conduct an extensive set of experiments to demonstrate that FogSurv using AI and data fusion help provide near real-time inferences and SA. Experimental results demonstrate that FogSurv provides a latency improvement of 37% on average over cloud architectures for the selected benchmarks. Results further indicate that combining AI with data fusion as in FogSurv can provide a speedup of up to
$9.8\times $
over AI without data fusion while also maintaining or improving the inference accuracy. Additionally, results show that AI combined with fusion of different image modalities obtained through UAVs in FogSurv results in improved average precision of target detection for surveillance as compared to AI without data fusion for different target scales and environment complexity.