EyeNet++: A Multiscale and Multidensity Approach for Outdoor 3-D Semantic Segmentation Inspired by the Human Visual Field

In processing dense 3-D point cloud data for semantic segmentation acquired by 3D sensors such as light detection and ranging (LiDAR), current deep learning networks face challenges in maintaining a sufficient neighborhood search area, limiting their accuracy and scalability. Inspired by the human visual field, EyeNet++ introduces a novel multiscale and multidensity input (MI) strategy, ensuring extensive spatial coverage of a neighborhood search area while preserving detail. The network is designed to maximize feature extraction from these diverse inputs through innovative modules. Parallel feature processing, facilitated by the messenger fusion (MF) block, enables the efficient exchange of complementary features between input scales. The dense local feature aggregation (DLFA) block enhances feature learning by expanding receptive fields and capturing local and global context. At the same time, the feature merge block seamlessly integrates features from multiple scales and densities for comprehensive and precise representation. Validated on large-scale benchmark datasets, such as SensatUrban, Toronto3D, Dayton Annotated LiDAR Earth Scan (DALES), York University Toronto Ontario Semantic (YUTOSemantic), and Semantic3D, EyeNet++ achieves state-of-the-art performance on SensatUrban, Toronto3D, and YUTOSemantic, effectively addressing the challenges posed by dense point clouds in outdoor scenes with various object sizes. The source code is available at https://github.com/Yacovitch/EyeNet2