A Semantic Segmentation Network Simulating the Ventral and Dorsal Pathways of the Cerebral Visual Cortex

Aiming at the problem of spatial information loss in the semantic segmentation process, we propose a semantic segmentation network, termed the ventral and dorsal network (VDNet), which simulates the ventral and dorsal pathways of the cerebral visual cortex. The ventral pathway network focuses on extracting semantic information, and the dorsal pathway network focuses on extracting spatial information. We use the semantic enhancement module (SEM) in the ventral pathway network to fuse information of different scales to enhance the extraction of semantic information, and we use the spatial attention module (SAM) in the dorsal pathway network to assign weights to different locations in space to enhance the extraction of spatial information. By fusing the information of the two pathways, the final semantic segmentation result is obtained. Since the dorsal pathway network is used to specifically enhance the extraction of spatial information, the problem of spatial information loss during the segmentation process is effectively improved, and higher segmentation accuracy can be achieved by using only a small backbone network. On the CamVid, Cityscapes and PASCAL VOC 2012 datasets, we achieve the mean intersection over union (mIoU) of 82.1%, 77.8%, and 81.0%, respectively, which verifies the effectiveness of the proposed method.