Signaling of object masks with the assistance of the object mask information SEI message

To reduce the decoder burden of performing the video analysis tasks and increase the accuracy of the task results, an effective solution is to perform the video analysis task in the encoder side and send the results to the decoder. In this paper, the object masks which are obtained by invoking a video analysis model in the encoder are represented by object mask pictures and encoded as auxiliary pictures. To enable this idea, a new auxiliary picture type was specified and a new supplemental enhancement information (SEI) message, the object mask information (OMI) SEI message, is introduced. The experiments are conducted by using versatile video coding test model (VTM) layered coding, and according to the experimental results, it is concluded that coding the object mask pictures as auxiliary pictures is feasible and efficient. And thus, the proposed OMI SEI message was adopted to working draft of versatile supplemental enhancement information version 4 and included in the technical report of optimization of encoders and receiving systems for machine analysis of coded video content in January 2024.