Mars Terrain Semantic Segmentation using Zhurong Rover Imagery Based on Transfer Learning of Historical Mission Data

Planetary rover has played an important role in deep space exploration. Terrain semantic analysis is of great significance to autonomous navigation and obstacle avoidance of Mars rover in exploration missions. However, it is difficult to build a semantic map of the environment around the Zhurong Mars rover due to the lack of semantic segmentation training data in landing area. This paper proposed a semantic segmentation method based on the combination of Martian historical mission data and simulation Mars rover data, which achieved the semantic segmentation of images obtained by Zhurong Mars Rover through PointRend based deep learning approach. The contributions of this paper can be mainly summarized in three aspects: (1) We proposed a Mars terrain semantic segmentation strategy based on knowledge transfer. (2) By combining historical exploration data and simulation data, the semantic segmentation model is applied to distinguish Martian landforms. (3) Refer to the landforms on the surface of Mars, we build a virtual Mars scene based on the simulation engine which generates the Mars scene simulation data with pixel level semantic tags. This paper evaluates the real image data of Zhurong rover, and experiments show that the method proposed in this paper can achieve terrain semantic segmentation with overall accuracy of 98.33% using images from Zhurong rover.