Integrating Intra-Phase Coherence and Intra-Frame Scene Perception for Panoptic Segmentation in Cataract Surgery Video

Panoptic segmentation is fundamental for the development of computer-assisted surgical techniques. Previous works have translated the video object segmentation task into a two-dimensional image segmentation task, that suffers from the interference of instruments with a similar appearance from different surgical phases. In this study, we propose to perform segmentation in a setup wherein the divided surgical phases are considered as separate video sequences, such that the model focuses only on the coherent features of the instruments appearing in the current surgical phase. The edge reconstruction branch is introduced to explore the intra-frame detailed features to enhance the structural perception of surgical scenes for maintaining the segmentation integrity. Considering the ambiguous edges commonly found in surgical scenes, we propose herein an entropy-guided edge reconstruction loss as the constraint. Extensive experiments are performed on a public cataract surgery video benchmark to demonstrate the superiority of our method.