On-Device Implementation of JML Based Decentralized Data Fusion with Non-Permissive Communications

There is an increasing demand for unmanned systems to perform a wide range of Intelligence, Surveillance and Reconnaissance (ISR) missions. The demand for small unmanned air systems (SUAS) results from reducing costs, increasing use for stand-off monitoring, and replacing piloted aircraft in disaster responses. With reduced size come more restrictive payload weights, limiting the number and diversity of sensors that can be located on a single platform. As a result, platforms must coordinate to share and leverage multi-sensor data over limited communications channels. This paper presents an on-device design and implementation of a distributed joint manifold learning (DJML) approach for improved object detection, classification, and identification (DCI). The DJML design addresses the joint utilization of sensor data from a collection of decentralized, heterogeneous sensing platforms (e.g., sensor swarm) in dynamic environments with constrained communications. This paper focuses on the implementation and evaluation of the hardware tradeoffs for distributed device mechanisms: drone-carried sensing, communication, and computing. Different sensor modality data are preloaded on SD (secure digital) cards and sequentially processed on local devices to emulate the additional sensor modalities. The in-lab testing results demonstrate the robustness and resiliency of on-device decentralized DCI under various conditions of sensor placement and communication degradation between platforms.