Gesture Recognition and Flight Control Using a Drone Camera

Recently, rapid advancements in drone technology have led to wider applications across various fields. Small drones, in particular, are lightweight and easy to operate. As a result, drone user interfaces are evolving—from previous controllers to more intuitive control methods using body movements. Currently, drones are mainly controlled using dedicated controllers or smartphone apps. However, these methods require users to learn stick and button operations, which can be difficult for beginners. Also, holding a controller for long periods can cause fatigue. To address these issues, this study focuses on face and gesture recognition to enable drone control without the need for a controller. We propose an integrated control system using a toy drone’s onboard camera, combining face and hand gesture recognition. Face recognition helps the drone automatically follow the user, keeping them within the camera’s view and improving usability. The system uses a Haar Cascade model for face detection and a CNN-based model for hand gesture recognition. These models are integrated into a control algorithm that prioritizes recognized gestures while tracking the user’s face. To ensure real-time performance, we used a lightweight face recognition model to allow face and gesture recognition to run in parallel.