Robust Depth-Based Person Re-Identification

Person re-identification (re-id) aims to match people across non-overlapping camera views. So far the RGB-based appearance is widely used in most existing works. However, when people appeared in extreme illumination or changed clothes, the RGB appearance-based re-id methods tended to fail. To overcome this problem, we propose to exploit depth information to provide more invariant body shape and skeleton information regardless of illumination and color change. More specifically, we exploit depth voxel covariance descriptor and further propose a locally rotation invariant depth shape descriptor called Eigen-depth feature to describe pedestrian body shape. We prove that the distance between any two covariance matrices on the Riemannian manifold is equivalent to the Euclidean distance between the corresponding Eigen-depth features. Furthermore, we propose a kernelized implicit feature transfer scheme to estimate Eigen-depth feature implicitly from RGB image when depth information is not available. We find that combining the estimated depth features with RGB-based appearance features can sometimes help to better reduce visual ambiguities of appearance features caused by illumination and similar clothes. The effectiveness of our models was validated on publicly available depth pedestrian datasets as compared to related methods for re-id.

I. Introduction

The task of person re-identification (re-id) is to match people in a distributed multi-camera surveillance system at different time and locations, with wide applications to forensic search, multi-camera tracking and access control, etc. In most short-term applications, low-level features such as color and textures are important appearance cues used to match. It is apparent that lighting will significantly affect the performance of these low-level features. In more extreme cases, when lighting condition changes greatly (e.g., with v.s. without lighting), color information of clothes becomes unreachable. Moreover, when people change clothes, color and textures become unreliable. For example, Figure 1 shows how color histograms change when people change clothes or appear in extreme illumination. In these cases, most existing re-id systems are not workable, since they are RGB-based. Fig. 1.

Illustration of change of color histograms and invariance of depth and skeletons. From left to right, the first column shows RGB images, the second column shows depth images (shown by pseudo-color images) and skeletons, and the remaining columns show histograms of R, G, B channels, respectively. (a) Clothing change. (b) Illumination change.