Learning Extreme Hummingbird Maneuvers on Flapping Wing Robots

Biological studies show that hummingbirds can perform extreme aerobatic maneuvers during fast escape. Given a sudden looming visual stimulus at hover, a hummingbird initiates a fast backward translation coupled with a 180-degree yaw turn, which is followed by instant posture stabilization in just under 10 wingbeats. Consider the wingbeat frequency of 40Hz, this aggressive maneuver is carried out in just 0.2 seconds. Inspired by the hummingbirds' near-maximal performance during such extreme maneuvers, we developed a flight control strategy and experimentally demonstrated that such maneuverability can be achieved by an at-scale 12-gram hummingbird robot equipped with just two actuators driving a pair of flapping wings up to 40Hz. The proposed hybrid control policy combines model-based nonlinear control with model-free reinforcement learning. We used the model-based nonlinear control for nominal flight conditions where dynamic models are relatively accurate. During extreme maneuvers when the modeling error becomes unmanageable, we use a model-free reinforcement learning policy trained and optimized in simulation to 'destabilize' the system for peak performance during maneuvering. The hybrid policy manifests a maneuver that is close to that observed in hummingbirds. Direct simulation-to-real transfer is achieved, demonstrating the hummingbird-like fast evasive maneuvers on the at-scale hummingbird robot.