REMIX: Real-Time Hyperspectral Anomaly Detection for Small UAVs

Unmanned aerial vehicles (UAV) have emerged in recent years as powerful, maneuverable sensors capable of real-time computer vision. Real-time image processing onboard UAV often requires data or model compression, acceleration, or edge offloading and is generally restricted to conventional RGB cameras. In this study, we consider real-time in-situ processing for hyperspectral imaging (HSI). HSI cameras detect many wavelengths of light. Material-specific spectral signatures can be matched to camera outputs to identify materials in a UAV's environment, but HSI cameras produce large amounts of information that generally require offline processing by heavy-weight software. We present REMIX, a real-time hyperspectral processing payload for small UAV. REMIX uses a custom software library, light-weight hyperspectral camera, and small embedded device to process and visualize HSI data in real-time. REMIX processes HSI lines in under 5 ms, allowing HSI perception to be visualized in real-time where conventional methods may take hours. We show that, when properly configured, adding real-time processing via REMIX degrades UAV flight time by only 4 % and increases HSI processing speeds by up to 6 X compared to naive payloads, and further decreases post-processing time by 20.48 X compared to conventional methods, even when using significantly less powerful equipment.