Elevator button tracking and localization for multi-storey navigation

Elevator button recognition in an indoor multi-storey environment has been a challenging task amidst the whole scenario of indoor navigation on a mobile robot. In this paper, we integrate various computer vision approaches for the task of button recognition and tracking in an indoor multi-storey environment. To overcome the problem of detecting elevator buttons, we have prepared a framework that uses various preprocessing techniques combined with object detection and tracking approaches to recognize the buttons. Initially, a single-shot object detector YOLOv3 locates the original positions of the target buttons using region over intersection based approach to produce bounding boxes over the required objects. Then we use a part-based tracking algorithm Deep-SORT that follows the detected buttons in realtime to counter the hard movements of the camera. lastly, we take the bounding box coordinate information of the detected buttons and make a semantic map, which can be used to recreate a complete layout of the button panel even with partially detected buttons or a frame consisting of partial button information.