Fusion and classification of aerial images from MAVS and airplanes for local information enrichment

Despite the existence of various matching algorithms, matching of images from Micro Aerial Vehicles (MAVs) and airplanes is still a tough problem due to the substantial differences in scale and rotation. This paper investigates the fusion of MAV imagery and airplane imagery and proposes a new robust image matching method with self-adaption to differences in scale and viewing direction. This method is further applied to register a MAV image block with reference to the orthophoto and DSM of a previously-geolocalized aerial image dataset. After registration, a fused 3D point cloud is generated and then combined with images as inputs for land cover (here roofs) classification. Experiments show that the proposed matching method outperforms SIFT/ASIFT methods in both quantity and reliability of matching results, while the registration of MAV imagery achieves decimeter-level accuracy without using any onboard GPS/IMU data. Besides, the pixel-level classification that integrates information of point clouds and images achieves significantly higher accuracy than simply image-based classification.