Uncovering Suspicious Activity From Partially Paired and Incomplete Multimodal Data

Multimodal data can be used to gain additional perspective on a phenomenon. For applications, such as security and the detection of suspicious activity, the need to aggregate and analyze data from multiple modes is vital. Recent research in suspicious behavior detection has introduced methods for identifying and scoring dense blocks in multivariate tensors, which are consistent indicators of suspicious activity. None yet, however, have proposed a method for the merging and analysis of multiple modes of data for suspicious behavior, especially when the set of items described in each data set do not match-that is, the data is partially paired-which is common when data sets originate from different sources. Neither has a method been described for dealing with the similar case of incomplete data. This paper introduces a technique for multimodal data analysis for suspicious activity detection when the data are only partially paired and/or incomplete. The method is applied to synthetic and real data, demonstrating strong precision and recall even in poorly paired cases.