A Joint Ship Detection and Waterway Segmentation Method for Environment-Aware of USVs in Canal Waterways

The canal waterways of China still play an important role in the logistics and transportation industry. Unmanned technology helps to reduce costs and improve the safety of navigation. Real-time environmental awareness is vital to making unmanned surface vehicles (USVs) come true. This paper proposes a new lightweight environmental awareness method based on deep convolutional neural networks (
DCNN
) and a mixed attention mechanism for USVs in canals, which can simultaneously perform ship detection, segmentation, and surface and background segmentation tasks. The features of the ships, surface, and background are extracted by a shared feature extraction backbone network and hybrid attention mechanism, which improves the efficiency of visual environmental awareness. In addition, a dataset named
USV-Canal
is constructed to enrich the features of canal waterways for environmental awareness, which contains typical canal scenes and 3443 ship objects. To improve the generalization, multiple public datasets are mixed with the
USV-Canal
dataset to build an integrated dataset to train our model, which boasts diversity in scene types and ship classes. The comparative and field experiments’ results show that 40.9% of
mAP
, 95.8% of
mIoU,
and 5 frames per second (
FPS
) inference speed can be achieved, and have good generalization, which can meet the requirements of environmental awareness of low-speed ships in canal waterways
Note to Practitioners
—The trained and validated model can ultimately be deployed on unmanned surface vehicles, and the required hardware platform is NVIDIA’s Jetson Nano, which is used for real-time perception of surrounding ships and navigable surfaces during navigation. The information can be integrated into the guidance, navigation, and control (GNC) system of USVs, achieving obstacle avoidance and ensuring safe navigation. It is vital to make autonomous navigation come true.