Coarse-to-Fine Visual Object Catching Strategy Applied in Autonomous Airport Baggage Trolley Collection

In this study, a systematic solution for mobile manipulation with the application of autonomous trolley collection in the airport is presented. The developed system features a novel integrated hardware and software system, incorporating a specially designed manipulator capable of tightly catching the trolley. Particularly, we innovatively present a supervisory control guided coarse-to-fine visual trolley catching strategy in large-scale airport environments. At the coarsest level, an image-based visual servoing method is utilized to approach the trolley from far to near, where a learning-based object detection method is adopted to identify trolleys in the environment in real-time. At the finest level, we present an accurate point cloud registration based object pose estimation method to promote the object manipulation with a position-based visual servoing controller. The tightly integrated system and the subsystems are evaluated through conducting experiments in the trolley collection task and comparing the key module with the state of the art. Extensive experiments demonstrate that the proposed framework can find and fetch the trolley robustly. Notably, the proposed object detection and pose estimation methods exhibit higher accuracy, providing credible perceptual approaches for mobile manipulation. Moreover, the proposed framework can be easily extended to other mobile manipulation tasks for object search and retrieval.