Visual Prompt Aided Single Shot Object Part Segmentation

Few Shot or Single Shot Segmentation of an object part is a challenging problem owing to various factors involving scarcity of part segmentation labels and varying scale and pose of the labeled object part (support-part) in relation to query object whole segment (query-whole). Prior attempts have been made to segment the query object part using additional language guidance with object part labels. Owing to lack of proper definition of an object part or lack of part label descriptions in many scenarios (e.g. an engine part), many real-world situations demand segmenting object parts with guidance from only a visual prompt. We propose to solve the problem of few shot part segmentation using only visual prompt by modeling it as a graph labeling problem. To this end, we introduce a novel super-pixel guided DiNOv2 feature based graph modeling for the whole and part of the support object (visual prompt) and the whole of the query object. We rely on an iterative graph partitioning strategy based on reverse guidance from query to support and converge to the most optimal query part segmentation. We showcase the efficacy of our proposed method in the most exhaustive part segmentation database, ADE20K-Part-234.