3-D Head Model Retrieval Using a Single Face View Query

In this paper, a novel 3D head model retrieval approach is proposed, in which only a single 2D face view query is required. The proposed approach will be important for multimedia application areas such as virtual world construction and game design, in which 3D virtual characters with a given set of facial features can be rapidly constructed based on 2D view queries, instead of having to generate each model anew. To achieve this objective, we construct an adaptive mapping through which each 2D view feature vector is associated with its corresponding 3D model feature vector. Given this estimated 3D model feature vector, similarity matching can then be performed in the 3D model feature space. To avoid the explicit specification of the complex relationship between the 2D and 3D feature spaces, a neural network approach is adopted in which the required mapping is implicitly specified through a set of training examples. In addition, for efficient feature representation, principal component analysis (PCA) is adopted to achieve dimensionality reduction for facilitating both the mapping construction and the similarity matching process. Since the linear nature of the original PCA formulation may not be adequate to capture the complex characteristics of 3D models, we also consider the adoption of its nonlinear counterpart, i.e., the so-called kernel PCA approach, in this work. Experimental results show that the proposed approach is capable of successfully retrieving the set of 3D models which are similar in appearance to a given 2D face view.

I. Introduction

While the previous focus on content-based multimedia retrieval is on the characterization and representation of entities like images and videos for query processing, more and more attention is being directed to the possibility of developing a content-based 3-D model retrieval approach, in which queries in the form of 3-D models are used to search for similar models in a database [1]–[3], [6]. The successful development of such an approach will have an important implication in application areas such as virtual world construction and game design, in which 3-D models can be rapidly constructed based on a set of pre-assembled components, instead of having to generate each model anew [3], [11], [24]. In view of this emerging requirement, we propose a new content-based retrieval approach for 3-D human head models, which form an important sub-class of 3-D models, in view of its relevance in many fields such as computer animation and medical imaging [4], [5], [28], [29]. However, it is not usually the case that a complete 3-D head model is available as query to the system, and the most distinguishing characteristic of our approach is that 2-D face views, instead of 3-D models [5], [29], are used as queries. This is in view of the difficulties of obtaining 3-D head models in practice, while 2-D portraits are amply available.