PanoSCU: A Simulation-Based Dataset for Panoramic Indoor Scene Understanding

Panoramic images offer a comprehensive spatial view that is crucial for indoor robotics tasks such as visual room rearrangement, where an agent must restore objects to their original positions or states. Unlike existing 2D scene change understanding datasets, which rely on single-view images, panoramic views capture richer spatial context, object relationships, and occlusions—making them better suited for embodied artificial intelligence (AI) applications. To address this, we introduce Panoramic Scene Change Understanding (PanoSCU), a dataset specifically designed to enhance the visual object rearrangement task. Our dataset comprises 5,300 panoramas generated in an embodied simulator, encompassing 48 common indoor object classes. PanoSCU supports eight research tasks: single-view and panoramic detection, single-view and panoramic segmentation, single-view and panoramic change understanding, embodied object tracking, and change reversal. We also present PanoStitch, a training-free method for automatic panoramic data collection within embodied environments. We evaluate state-of-the-art methods on panoramic segmentation and change understanding tasks. There is a gap in existing methods, as they are not designed for panoramic inputs and struggle with varying ratios and sizes, resulting from the unique challenges of visual object rearrangement. Our findings reveal these limitations and underscore PanoSCU’s potential to drive progress in developing models capable of robust panoramic reasoning and fine-grained scene change understanding.