Fuzzy Integral Sliding Mode Observer-Based Structure and Motion Estimation of Multiple Objects Using a Monocular Camera

This article proposes a method to estimate the structure and motion (SaM) of multiple objects using a monocular camera. Most of the existing methods for SaM estimation are restricted to a single object by constraining the camera and object motions. In this study, SaM estimation is extended for the case of multiple (or simply two) objects, and the restrictions on the camera and objects motions and the necessity for the information related to the camera velocity are relaxed by using the relative information of the two objects. The relative motion dynamics of partially measurable states of multiple objects are first introduced based on which the ranges (i.e., depths) of the two objects with respect to the camera, relative velocity between the objects, relative velocity between each object and the camera, and velocities of both the camera and objects are then estimated by using the proposed fuzzy integral sliding mode observer. The following cases for the camera and objects—that have static (stationary), dynamic (moving), or general (i.e., either static or dynamic) motions—are studied: 1) Two objects are at the same range with respect to the camera and 2) two objects are at different ranges with respect to the camera. Several situations—in which the ranges of the two objects with respect to the monocular camera, relative velocity between the two objects, and velocities of the camera and objects can be estimated—are described in detail. Simulations and experimental results demonstrate the validity of the proposed method.