NeighborLoss: A Loss Function Considering Spatial Correlation for Semantic Segmentation of Remote Sensing Image

House segmentation of remote sensing image based on deep learning has become the main segmentation method because it can automatically extract features. However, the accuracy of image segmentation is affected not only by the network model, but also by the loss function, but the existing loss functions, except Binary Cross Entropy, are designed to deal with imbalanced dataset, no new research on improving Binary Cross Entropy for balanced dataset, and all loss function treat each pixel in isolation, without considering the spatial correlation between pixel and its neighbor pixels. To solve this problem, a new loss function, named NeighborLoss function, is proposed. Firstly, the deep learning network is used to get the prediction results of each pixel. According to whether the prediction results of the eight neighboring pixels of each pixel are consistent with the each pixel prediction, different weights are given to each pixel. Finally, the weighted average value of cross entropy of all pixels in the batch is taken as the final loss function value. We use the main deep learning semantic segmentation networks SegNet, PSPNet, UNET ++, MUNet with both NeighborLoss and cross entropy loss respectively to extract houses on the open data set named WHU dataset for remote sensing. The results show that compared with cross entropy loss functions, the MIoU, Precision, Recall, and Accuracy of NeighborLoss function are improved. From the predicted graph, the NeighborLoss function is more accurate to extract the edge of the house, especially in the corner of the house. NeighborLoss function is a more effective loss function for remote sensing image segmentation.