A Promptable Segmentation Approach to Automatic Floor Plan Analysis using Vision Transformers

Automatic floor plan analysis, rooted in computer vision and pattern recognition, aims to extract critical insights from architectural and interior design drawings, with applications in architecture, real estate, and construction. Conventional methods, whether rule-based or learning-based, face limitations, including extensive manual annotation requirements and challenges in achieving precise segmentation. This research study evaluates the efficacy of Vision Transformers (ViTs) for key floor plan analysis tasks, such as identifying structural elements like doors and windows, extracting line segments to represent room layouts, and segmenting regions like room spaces. We introduce a novel promptable segmentation approach that integrates object detection and image segmentation to achieve a comprehensive understanding of scenes. This methodology centers on the Segment Anything Model (SAM), utilizing input prompts generated by object detection models to guide segmentation. For generating these prompts, we leverage real-time object detection models, such as You Look Only Once Version 8 (YOLOv8) and end-to-end object detection models such as Real-Time Detection Transformer (RT-DETR). Experimental results across diverse datasets demonstrate robust performance, with high Mean Average Precision (mAP), Precision, and Recall rates, setting a new benchmark for automatic floor plan analysis.