Towards Nano-Drones Agile Flight Using Deep Reinforcement Learning

Recent advancements in Deep Reinforcement Learning (DRL) have paved the way for novel strategies in developing intelligent autonomous quadrotors, renowned for their agility and versatility. However, obtaining agile and robust flight policies that can be deployed in real hardware is still a main issue in enabling the spread of those algorithms. Moreover, deploying those policies in resource-constrained nano-drones is often unfeasible. In this paper, we address the following research problems: i) How to train a DRL policy in a dynamic environment to perform an agile flight task? ii) How does this flight strategy behave against state-of-the-art model-based methods? iii) Is implementing this neural network on a nano-drone equipped with ultra-low power computing resources feasible? To address them, we have effectively trained a Multilayer Perceptron (MLP) using the Proximal Policy Optimization (PPO) algorithm within Isaac Gym parallel simulator. We compared the MLP performance against a state-of-the-art Model Predictive Control (MPC) approach enhanced with learned high-level policy in solving a highly dynamic flight task. Our approach improved the success rate in challenging environment scenarios by up to 95%. We evaluated our neural network policy inference latency and memory footprint on the AI deck, a companion computer of the Crazyflie nano-drone based on GAP8, a constrained and ultra-low power System-on-Chip (SoC). We achieved an inference latency of 2.5
$\mu s$
, making it suitable for real-time control actions. The proposed work proves the possibility of using Deep Neural Networks (DNNs) to encode complex control strategies for resource-constrained robotic platforms.