3D Deep Object Recognition and Semantic Understanding for Visually-Guided Robotic Service

For the success of visually-guided robotic errand service, it is critical to ensure dependability under various ill-conditioned visual environments. To this end, we have developed Adaptive Bayesian Recognition Framework in which in-situ selection of multiple sets of optimal features or evidences as well as proactive collection of sufficient evidences are proposed to implement the principle of dependability. The framework has shown excellent performance with a limited number of objects in a scene. However, there arises a need to extend the framework for handling a larger number of objects without performance degradation, while avoiding difficulty in feature engineering. To this end, a novel deep learning architecture, referred to here as FER-CNN, is introduced and integrated into the Adaptive Bayesian Recognition Framework. FER-CNN has capability of not only extracting but also reconstructing a hierarchy of features with the layer-wise independent feedback connections that can be trained. Reconstructed features representing parts of 3D objects then allow them to be semantically linked to ontology for exploring object categories and properties. Experiments are conducted in a home environment with real 3D daily-life objects as well as with the standard ModelNet dataset. In particular, it is shown that FER-CNN allows the number of objects and their categories to be extended by 10 and 5 times, respectively, while registering the recognition rate for ModelNet10 and ModelNet40 by 97% and 89.5%, respectively.