Real-Time Digital Video Stabilization Based on IMU Data Fusion and Periodic Jitters of Airborne Vision of Bionic Flapping Wing Robots

Bionic flapping wing robots (FWRs) with airborne vision return real-time video streams during flight for critical tasks such as aerial surveys, traffic patrols, and fire warnings. Periodic wing flapping, however, introduces intense jitters into the monitoring video streams. A real-time digital video stabilization (DVS) method based on inertial measurement unit (IMU) data fusion and periodic jitters, which can be operated in real time via an airborne computer and return stabilized video streams to the ground station with low latency, was proposed in this study. First, multiple continuous trajectories with equal spatial intervals were generated on the basis of airborne IMU data and optical flow tracking. Second, the lagged stable positions and velocities of the trajectories were estimated on the basis of the periodic characteristics and frequency of the flapping wings via an airborne Hall sensor. Then, the stable trajectories of the current frame were predicted with a constant-velocity (CV) model and guided to generate stable frames. Finally, outdoor flight experiments were conducted with a 1.8 m wingspan FWR. In terms of objective and subjective evaluations, the proposed method provides greater stabilization performance and robustness for different scenes and FWR flight status than existing methods do.