ENSURE: A Deep Learning Approach for Enhancing Situational Awareness in Surveillance Applications With Ubiquitous High-Dimensional Sensing

Judicious processing of
High-Dimensional Sensing (HDS)
data streams is often the first step in a data-driven decision-support system. However, how one goes about processing large amounts of incoming HDS data streams for enhancing
situational awareness (SA)
is a non-trivial, and often challenging task due to the interwoven nature of operational environments, especially in complex surveillance applications (
e.g.,
predictive policing, urban reconnaissance). Toward addressing this challenge, a novel data processing framework called
ENSURE (ENhanced Situational Understanding with Ubiquitous-sensing REsources)
is presented. In particular, by using
density-based clustering
in tandem with variable length sequence decoding methods, ubiquitous HDS data streams are clustered and then processed for early identification of ‘events’ and their sequences of interest to a (surveillance) task at hand. Then, by leveraging notions in
data association,
ENSURE ‘tracks’ the occurrence of such event sequences and their evolution over time to ‘perceive’ and ‘comprehend’ what’s happening in the operational environment. Therefore, ENSURE can be interpreted as an
information fusion
approach for HDS systems for object recognition, classification, tracking, and decision-making tasks. Use of ENSURE is illustrated via a simulated experiment to provide insights into its behavior, performance and sensitivity to parameters.