A Feature Level Multi-Sensor Information Fusion Strategy for Land Region Classification

Deep learning has seen success in image recognition and classification in remote sensing research areas. Specifically, it is capable of classifying land coverage by its usage, such as determining if a plot of land is a residential area, industrial area, forest, etc. This is useful for quickly parsing datasets for surveillance applications and also for autonomously categorizing municipal land for urban planning. However, current methods learn and classify based on a single data stream (EO) and consequentially a single set of features. Our approach for land classification, LandNet, uses a feature-level fusion approach of both EO and LiDAR data in a deep learning classification system. In our approach, we use an aerial dataset of geo-registered scenes captured by both EO and LiDAR sensors. We propose a deep-learning based feature-level fusion framework for integrating information extracted from 2D and 3D data. In our preliminary testing, we observed that this proposed network is a more effective approach as it employs feature sets of multiple sensors, and it outperforms each of the individual modalities.