MAE-PixelSeg: Fine-Grained Urban Land Cover Classification with Self-Supervised Transformer

Land use land cover classification (LULC) plays a pivotal role in comprehending and managing the Earth’s surface. Numerous efforts have concentrated on deep learning methods utilizing supervised techniques, but their performance often relies on extensive well-annotated data, incurring significant time and labour expenses. This paper introduces MAE-PixelSeg, a novel self-supervised framework for LULC, leveraging a worldwide dataset of unlabeled Sentinel-2 satellite images from the Google Earth Engine (GEE) platform. MAE-PixelSeg employs the Vision Transformer (ViT) backbone, initialized by the MAE encoder, with a Shuffle Neck for high-resolution hierarchical feature map extraction and an ASPP Head for precise segmentation. Results demonstrate MAE-PixelSeg’s superiority over baseline methods, achieving a remarkable mean Intersection over Union (mIoU) of 71.07% on WorldCover and 77.43% on GID. Furthermore, we show that self-supervised pre-training yields robust feature representations, enabling commendable performance when transferred to other datasets, particularly in situations with severely limited annotated data.