iCaps: Iterative Category-Level Object Pose and Shape Estimation

This letter proposes a category-level 6D object pose and shape estimation approach iCaps, which allows tracking 6D poses of unseen objects in a category and estimating their 3D shapes. We develop a category-level auto-encoder network using depth images as input, where feature embeddings from the auto-encoder encode poses of objects in a category. The auto-encoder can be used in a particle filter framework to estimate and track 6D poses of objects in a category. By exploiting an implicit shape representation based on signed distance functions, we build a LatentNet to estimate a latent representation of the 3D shape given the estimated pose of an object. Then the estimated pose and shape can be used to update each other in an iterative way. Our category-level 6D object pose and shape estimation pipeline only requires 2D detection and segmentation for initialization. We evaluate our approach on a publicly available dataset and demonstrate its effectiveness. In particular, our method achieves comparably high accuracy on shape estimation.

I. Introduction

Estimating 6D object poses, i.e., 3D translations and 3D orientations of objects with respect to cameras, is crucial for a variety of real-world applications ranging from robotic navigation and manipulation to augmented reality and virtual reality. The majority of existing works have so far mainly dealt with the instance-level 6D pose estimation [1]–[7], where a set of 3D CAD models of known instances are given as priors. The problem is thereby reduced to finding the sparse or dense correspondence between a target object and a prior 3D model. Although 3D CAD models are available in some industrial applications such as assembling different parts, the requirement still significantly limits many practical robotic applications since it can be expensive or even impossible to acquire 3D CAD models of all the objects in an environment.