Dimensional Dilemma: Navigating the Fusion of Hyperspectral and Lidar Point Cloud Data for Optimal Precision – 2D vs. 3D

Despite the extensive body of research conducted on the fusion of lidar and hyperspectral data for land cover classification in urban areas, the predominant approach has been the utilization of rasterized lidar data merged with hyperspectral data. This image-centric methodology tends to overlook the primary advantage inherent in lidar technology—namely, the production of 3D point cloud data. In our work, we present a framework demonstrating how we infer semantic information from 3D point cloud data, comprising both lidar and hyperspectral features—a concept we refer to as a 3D hyperspectral point cloud. We illustrate the generation of hyperspectral point clouds and evaluate the performance of various deep learning models for point learning. Our findings on the original test data of the 2018 IEEE GRSS Data Fusion Challenge, disclosed by IEEE Image Analysis and Data Fusion Technical Committee, indicate that recent deep learning models not only produce better shapes for predicted objects but also yield more precise semantic information. Finally, we plan to release the 3D hyperspectral point cloud data to the community, hoping to inspire future studies on data fusion in the point cloud domain.