Semantic-Based Multi-Object Search Optimization in Service Robots Using Probabilistic and Contextual Priors

In recent years, the demand for service robots capable of executing high-level tasks has grown. In the future, service robots will be expected to perform complex tasks like ‘Set table for dinner’. Such high-level tasks require that the robot possess the ability to retrieve multiple objects from the environment. Thus this paper investigates the challenge of locating multiple objects in an environment, termed ‘Find my Objects’. In our approach, we present a novel model for extraction of ‘Environment-specific’ priors from generalized data available in public datasets. We present a novel heuristic specifically designed to optimize Multi-Object search in indoor spaces while considering User Preferences. We also propose a novel Post-task Position Optimization (PTPO) strategy for improved performance in successive tasks. PTPO enables the robot to leverage information gained during a task to improve its inferencing for the next task. Our approach is built on a Semantic Mapping framework that combines semantic object recognition with geometric data to generate a multi-layered map. We fuse the Semantic Map with environment-specific priors in our inferencing strategy. Importantly, our method is agnostic to object detectors, Visual SLAM techniques, and local navigation planners. We demonstrate the ‘Find my Objects’ task in real-world indoor environments, yielding quantitative results that attest to the effectiveness of our methodology. This strategy can be applied in scenarios where service robots need to locate, grasp, and transport objects, taking into account user preferences.