UAV-based Foliage Plant Species Classification for Semantic Characterization of Pre-Fire Landscapes

In this work we deal with the problem of establishing a system architecture to facilitate the real-time autonomous volumetric mapping alongside the semantic characterization of sagebrush ecosystem landscapes, in order to support the pre-fire modeling and analysis required to plan for wildfire prevention and/or suppression. The world, and more specifically the broader region of N. Nevada has been facing one of its most challenging periods over the course of the last decade, as far as uncontrolled wildfires are concerned. This has led to the development of research initiatives aimed at the ecosystem-specific modeling of the pre-, during-, and post-fire process effects in order to better understand, predict, and address these phenomena. However, to collect the required wide-field information that contains both centimeter-level volumetric mapping fidelity, as well as semantic details related to plant (sub)-species, which for the common case of sagebrush can only be identified based on close-up inspection of their foliage fine structure, satellite photography remains insufficient. To this end, we propose a perception and mapping architecture of an aerial robotic system that is capable of: a) LiDAR-based centimeter-level reconstruction, b) robust multi-modal sensor fusion Simultaneous Localization and Mapping (SLAM) lever-aging LiDAR, IMU, Visual-Inertial Odometry, and Differential GPS in a global optimization mapping framework, as well as c) a gimbal-driven point-zoom camera for the efficient real-time collection of close-up imagery of foliage pertaining to specific target plants, in order to allow their real-time identification based on their leaf micro-structure, by leveraging Deep-Learned classification deployed on a Neural Processing Unit. We present the associated systems, the overall hardware and software architecture, as well as a series of field deployment studies validating the proposed aerial robotic capabilities.