Fine-Grained Food Image Segmentation Method Based on MS-Mask2Former

In the context of the combination of Artificial intelligence of things (AIoT) and Food computing, food image segmentation is a current research hotspot. It plays an important role in food selection, intake and nutritional assessment. The main problems leading to food image segmentation errors are the presence of a large number of irregularly shaped objects in the same category and the overlapping or intersection of edge textures of multi-category objects. We propose a fine-grained food image segmentation method based on MS-Mask2Former to address these difficulties. On the basic network structure of Mask2Former, First, we designed the Multi-scale Enhanced Feature Pyramid Network(MEFPN). This module processes feature maps of different resolutions in multi-scale feature maps separately to capture global and local features, thereby expanding the receptive field of the network and extracting complex features of irregularly shaped foods. Secondly, we propose Semantic Concern (SC) to realize the positional prior. Guided by low-resolution feature maps, the SC module selects regions with richer semantic information from high-resolution feature maps to achieve clear segmentation of food objects and obtain distinct boundaries. The experimental results show that our MS-Mask2Former method improves 2.16% in terms of mIoU, 2.37% in terms of mAcc, and 1.18% in terms of aAcc compared to other representative and excellent methods in the field of food segmentation, demonstrating excellent segmentation performance and achieving SOTA performance on the FoodSe2103 dataset.