Mobile Robot Place Segmentation and Categorization Using Object Semantics

A mobile robot’s ability to perform complex tasks depends critically on its understanding of the environment in which it operates. This understanding—often represented as a map—must incorporate both semantic information about objects (identity, functionality, …) and a high-level grasp of spatial organization. This latter capability, referred to as place segmentation and categorization, is essential for reasoning in diverse and dynamic spaces. However, conventional semantic mapping methods typically segment environments into predefined rooms and rely on closed-set labels, limiting adaptability and expressiveness in real-world, multi-functional settings.In this paper, we propose a novel pipeline for indoor place segmentation and categorization that leverages rich object semantics to generate more flexible and meaningful spatial representations. Rather than adhering to rigid architectural boundaries, our approach defines places as clusters of functionally coherent objects. These clusters are formed using descriptors that integrate geometric location with semantic information, derived from context-aware functional descriptions of objects generated by Large Language Models (LLMs) and encoded as sentence embeddings. We apply density-based clustering to identify semantically meaningful places and use LLMs to assign each cluster open-set natural-language tags and detailed functional descriptions, resulting in enriched and interpretable maps.Experimental results on the ScanNet and SceneNN datasets demonstrate that our method significantly improves segmentation consistency over geometric and basic semantic baselines while producing coherent and informative place categorizations that better reflect the functional layout of indoor environments.