Panoptic Water Surface Visual Perception for USVs using Monocular Camera Sensor

In recent years, the significance of unmanned surface vehicles (USVs) has grown substantially across a wide range of applications. Monocular cameras, as the most common perception sensors deployed in USVs, offer the inherent advantages of rich semantic information and low-cost deployment. However, visual perception methods for USVs face challenges from water surface application and lose efficiency in harsh weather cases. To achieve robust perception capabilities through monocular cameras for USVs, we propose a panoptic water surface visual perception framework that can accomplish various perception tasks, including drivable area segmentation, object detection and raindrop segmentation in water surface scenes. In addition, our framework provides a SAM-driven training method to improve the robustness of the proposed model through low-cost model iteration. Moreover, the proposed perception method demonstrates excellent accuracy, robustness and high inference speed compared to other light-weight baseline methods and can be deployed to low-power embedded platforms in USVs.