Enhancing Mask2Former for Real-Time Universal Human Image Segmentation

This study aims to speed up the method of image segmentation for humans, based on Mask2Former, to real-time. We propose the Multi-Fusion Model, which boosts the overall model speed by 50% while maintaining good result quality. Next, we employ a simplified Transformer Decoder that has been experimentally optimized. Additionally, we implement all preprocessing, post-processing, and result rendering on the GPU. We can render the segmentation results of 1080p resolution at a speed exceeding 600 FPS. The general image segmentation model can achieve a speed of over 40 FPS for detecting high resolution 1080p images.