IEEE Xplore At-A-Glance
  • Abstract

Intrinsic Geometric Scale Space by Shape Diffusion

This paper formalizes a novel, intrinsic geometric scale space (IGSS) of 3D surface shapes. The intrinsic geometry of a surface is diffused by means of the Ricci flow for the generation of a geometric scale space. We rigorously prove that this multiscale shape representation satisfies the axiomatic causality property. Within the theoretical framework, we further present a feature-based shape representation derived from IGSS processing, which is shown to be theoretically plausible and practically effective. By integrating the concept of scale-dependent saliency into the shape description, this representation is not only highly descriptive of the local structures, but also exhibits several desired characteristics of global shape representations, such as being compact, robust to noise and computationally efficient. We demonstrate the capabilities of our approach through salient geometric feature detection and highly discriminative matching of 3D scans.

SECTION 1

Introduction

Identifying robust features from geometric objects is of crucial importance in many areas, such as visualization, shape registration, and shape classification, to name a few. In order to automatically derive and analyze features from real-world measurements, scale space representation of signals are often employed to account for the scale variability of underlying structures [17]. A main intention behind it is to obtain a separation of the structures retained in the original data, such that fine structures only exist at finest scales in the scale space. Therefore, operations performed at certain scales will be simplified, provided that unnecessary and irrelevant fine-scale structures have been suppressed.

The rapid growth in the number and quality of geometric models and their ubiquitous use in a large number of visual computing applications, suggest the need for more powerful tools in 3D shape analysis. Having witnessed the success of scale space theory applied to image and image feature extraction [19], [21], people are naturally inspired to generalize a similar framework for 3D shapes [36], [8], [26], [23], [16]. For instance, Kimmel constructed a geometric scale space for images painted on a given surface [13]. In [26] and [16], point clouds are explicitly smoothed by means of mean curvature flow and least squares projection, respectively, for multiscale shape representations. Zou et al. [36] treated multiple geometric properties as an image retained on the surface, on which successive geodesic Gaussian smoothing is performed on the curved domain. A common characteristic of these methods is that, although labeled by different smoothing techniques, the original curved surfaces served as domains. Through surface parameterizations, scale-space shape representation and subsequent analysis can also be defined on the UV domain [23], [8], where surface geometry is modeled as a normal map in [23], and a vector image consisting of curvatures and area distortion factors in [8]. The UV domain is subsequently represented as 2D images. Since the image can easily attain a much higher resolution than triangular meshes, the geometric information is well preserved in the image representation with little loss. Furthermore, because of the regular structure of images, features extracted by this approach are generally more robust than those directly extracted from 3D meshes. Thus far, all scale-space representations were established at the measurement level, in which case, whether coarser scale representations correspond to a physically valid geometry is of little concern.

To further leverage the scale-space representations by the underlying geometric structures of 3D shapes, our proposed scale-space alternative is established upon the intrinsic geometry of 3D shapes. In sharp contrast to an extrinsic point of view, shapes are generalized as a 2-manifold, equipped with a Riemannian metric. This approach describes the geometry of a shape as an "insider". As a consequence, the intrinsic scale-space representation is independent of how the surface is positioned in ℝ3 namely, the embedding in ℝ3 This paper focuses on constructing a formal geometric scale space through intrinsic geometric diffusion. The correctness is fully verified through our formal proof. By integrating geometric comprehension into the scale-space representation, many properties are better understood and with further control. A lot of tasks in graphics and visualization can benefit from this computational model. In particular, the scale-aware geometric feature is presented as a natural application of this novel geometric scale space. As geometric features are organized according to their inherent scales, the resulted representation is not only highly descriptive of the local structures, but also compact, robust to noise and computationally efficient.

1.1 Prior Work

The Scale Space Theory was first formulated in the computer vision community based on 2D images [33]. Provided the presupposition that new structures must not be created from a fine scale to any coarse scale, the Gaussian kernel of increasing variance was selected to derive a one parameter family of smoothed images I(x, y, t), namely, the scale space. More mathematically, the same scale space can be equivalently derived as the solution of the heat diffusion equation [9], where the intensity value of the image was interpreted as a temperature distribution in the image plane and the former convolution with Gaussian kernel corresponds to the heat diffusion over time t. By allowing the diffusion coefficient to vary, an edge-preserving version of image scale space was derived from boundary detection and anisotropic diffusion [27]. More recently, scale space has been successfully applied to scale-invariant feature detection of images [17], [19], [21], [1]. However, a multi-scale representation by itself contains no clue about the scale information of the underlying structures. To avoid the instability of image descriptors computed at inappropriately chosen scales, Lindeberg [17] proposed an automatical scale selection methodology in the absence of prior knowledge about the images and, meanwhile, showed that certain normalized differential operators assume local extrema over scales which correspond to the characteristic sizes of respective structures in the image. In practice, the difference-of-Gaussian (DoG) function was often used for keypoint detection in scale space [19], [21], [1]. The frequency of sampling in the image and scale domains need be pre-determined with caution, which trades off efficiency and reliability with completeness.

3D shapes share the same multiscale nature as 2D images. Given the success of scale space approaches to the planar image, some recent work has been conducted, aiming at generalizing a similar framework to the geometry of 3D shapes. Along this direction, Kimmel constructed a geometric scale space for images painted on a given surface, using a level set method [13]. Mokhtarian et al.[22] applied this concept to free-form 3D object recognition, where a uniform optimal scale was empirically determined for the features of each model according to the number of rest feature points vs. the number of smoothing iterations. By resorting to the surface parametrization, both Hua et al. [8] and Novatnack et al. [23] represent 3D shapes as planar vector images. The scale space was consequently constructed on the geometric attributes retained on the image. For the point-based 3D shapes, a scale space formulated via surface variation was successfully applied to shape deformation [26], line-type feature extraction [25], approximate alignment [16], and 3D model segmentation [14]. However, as aforementioned, none of these methods explicitly related their proposed representations to the fundamental axiomatic properties of scale space theory. Therefore, it is desired that a formal analogue of scale spaces between 2D images and 3D shapes can be rigorously established.

On the other hand, the geometric flows [34], [10], [3], [35], [7], [29], [6] previously seemed a separate field which may be bridged by this paper. Here we merely attempt to show the inherent link between shape smoothing (scale space processing) and geometric flows. In [3], the mean curvature flow was employed for implicit mesh fairing. Because it only depends on the geometric properties of the mesh, consequent smoothing is triangulation-invariant. With some constraints and adjustment to the surface normal, the Gaussian curvature was also used for surface smoothing in [35]. Recently, the Ricci flow received much attention for its essential role in the proof of the Poincaré conjecture [7]. As the Gaussian curvature is induced by the Riemannian metric of the surface, diffusion of the metric will affect the Gaussian curvature accordingly. Surface Ricci flow has been demonstrated as a powerful tool in shape analysis [6]. Its known applications, however, are essentially limited to the conformal surface parametrization [29], [11].

In this paper, we are investigating its potential in the scale-space representation of 3D shapes. Unlike mean curvature flow [10] and others, the Ricci flow is performed purely on the intrinsic geometry of the surface shape as a process of metric diffusion. Surfaces deformed via Ricci flow are conformally equivalent, which preserves the geometric structures up to an isotropic scaling. Hence, we claim that it is the first scale space formulation with respect to the intrinsic geometry of 3D shapes.

SECTION 2

Intrinsic Geometric Diffusion

Given the criteria for a range of properties of the scale space (i.e., scale space axioms), the Gaussian smoothing constitutes the canonical way of generating a scale space. Equivalently, the scale-space sequence can be defined as the solution of a diffusion equation Formula TeX Source $$a\left| {p_i - p_j } \right|^\alpha + b\left| {s(p_i)} \right. - s(p_j)\left| {^\beta } \right..$$where L(0) is the original signal. In this section, we shall present the construction of our novel intrinsic geometric scale space (IGSS) through the surface Ricci flow—a diffusion of the intrinsic Riemannian geometry on the surface.

2.1 Surface Ricci Flow

Ricci flow is a powerful curvature flow method in geometric analysis. In brief, it conformally deforms the Riemannian metric on a surface according to the induced Gaussian curvature, such that the curvature evolves in a heat diffusion fashion. The Ricci flow only depends on the intrinsic surface geometry and the final target curvature, which makes it an excellent 3D shape representation in a broad range of potential applications [11]. Later, we will show that a scale space of surface geometry constructed via the surface Ricci flow satisfies a set of desired properties for a multiscale representation, conventionally referred to as the scale-space axioms.

Let S be a surface embedded in ℝ3; the induced Riemannian metric is denoted by g. Given a scalar function u : S → ℝ defined on Formula is also a Riemannian metric on S. Surfaces with metrics having this relation form a group of conformal equivalence (i.e., angle-preserving). The surface geometry is locally preserved up to a scaling factor. Surface Ricci flow deforms the metric g(t) over time t according to the current Gaussian curvature K(t): Formula TeX Source $${{dg(t)} \over {dt}} - 2K_g g(t),$$ such that the curvature evolves exactly following the diffusion equation Formula TeX Source $${{\partial K} \over {\partial t}} = - \Delta _{g(t)} K.$$ where Δg(t) is the Laplace-Beltrami operator under metric g(t). For a detailed deduction, refer to [29]. If we substitute the metric with g(t) = e2 u(t)g(0), Eq. (2) is equivalent to Formula TeX Source $${{du(t)} \over {dt}} = - K_u .$$As shown by Eq. (3), the Ricci flow is a formal analogue of the heat diffusion on the surface geometry. During the diffusive process, the Gaussian curvature evolves in the same manner as image intensity in the image scale space. Therefore, the properties that were assumed in the SIFT-like methods [19], [21] are naturally retained by the IGSS dealing with surfaces. Note that, the Gauss-Bonnet formulaSKdA+ ∫SKg ds = 2π χ (S), places the only constraint on the final curvature as well as the metric, where K is the Gaussian curvature Kg is the geodesic curvature along the boundary ∂SД(S) is the Euler characteristic number of S. To eliminate the scaling ambiguity, we further require that ∫udA = 0.

2.2 Theoretical Discretization

In practice, surfaces are often approximated using triangular meshes. To generalize the continuous Ricci flow to a mesh, we first define our notations. We denote a triangular mesh as a pair Formula, where τ is an abstract simplical complex which contains all the topological (adjacency) information, and G represents the geometric realization. The complex τ = (V E F) consists of a vertex subset V = {vi}, an edge subset E = {eij}, and a face subset F = {fijk}, where vi denotes the ith vertex eij the oriented edge from vi to vj with the length denoted as lij fijk the oriented face formed by vi vj vk in an order such that its counter-clockwise orientation gives the face normal pointing outside. The geometric realization G: V → ℝ3 embeds τ in ℝ3 Instead of casting a specific problem for a discrete numerical solution, we pursue a systematical translation of the continuous theory. Properties that are valid in the continuous setting can easily find analogous justifications in the discrete formulation.

From associating a circle to each vertex and making sure that two circles are tangent if and only if the corresponding vertices are connected by an edge, a configuration of these circles, namely, circle packing unites the combinatorics and the discrete geometry of a triangulation [32], [30]. It was proven that circle packing actually converges the classic conformal structure under infinite refinement [28], which shed light to a valid means to generalize the continuous conformal geometry to a discrete mesh [30]. As classical conformal geometry abstracts infinitesimal circles on the surface and concerns the transformations where only the "radius" changes, its discrete counterpart operates with real circles. Furthermore, the condition of tangency can be generalized to allow an intersection angle between neighboring circles, offering more flexibility in practice.

More specifically, each vertex vi is associated with a circle of radius γi, and each edge eij assigned a weight wij such that the equation lij2 = γi2 + γj2 + 2γi γj · wij holds. A mesh deformation is conformal if and only if edge weights are preserved constant. In fact wij is the cosine of the intersection angles when the neighbor circles have overlap, or the inversive distances between adjacent vertices when their associated circles do not intersect. Another typical definition of discrete metric of a triangle mesh is the edge lengths [29]. The circle packing metric is equivalent to the edge length metric in the sense that, given one, we can straightforwardly compute the other.

On the other hand, a triangle mesh is essentially a piecewise Euclidean surface. Each vertex is a cone singularity. Curvatures are concentrated at each vertex. Conventionally, the vertex curvature is defined as the angle deficit, Formula TeX Source $$K_i = \left\{ \matrix{2\pi - \sum\nolimits_{f _{ijk} \in F} {\theta ^{jk} } \quad {\rm{if}}\;v_i \;{\rm{is}}\;{\rm{an interior vertex;}} \hfill \cr \pi - \sum\nolimits_{f _{ijk} \in F} {\theta _i^{jk} } \quad {\rm{if}}\;v_i \,{\rm{is a boundary vertex}}{\rm{.}} \hfill \cr} \right.$$where θijk is the angle formed by edge eij and eki in the face fijk. For interior vertices Ki is called the discrete Gaussian curvature, while on the boundaries it is called the discrete geodesic curvature. Given the three sides lij ljk and lki of triangle fijk, the corner angle θjki associated with vi can be computed by the cosine formula: Formula TeX Source $$\theta _i^{jk} = \cos ^{ - 1} \left({{{l_{ij}^2 + l_{ki}^2 - l_{jk}^2 } \over {2l_{ij} l_{ki} }}} \right).$$Similar to the fact that the Gaussian curvature is determined by the metric for a smooth manifold, the discrete curvatures are only determined by the radii that appeared in the circle-packing structure. The Gauss-Bonnet Theorem still applies to a mesh, in the form of ΣviV Ki = 2π χ (M). Therefore, the surface Ricci flow is ready to be properly translated to the discrete case. Suppose the radii {γi} and the edge weights {wij} are given for mesh M. The discrete Ricci flow is defined at each vertex vi as Formula TeX Source $${{du_i } \over {dt}} = - K_i,$$where ui = ln γi. The discrete surface Ricci flow has the same form as the continuous case (See Eq. (4)). The correspondences of related concepts are summarized in Table 1. In order to approximate the original geometry as far as possible, ideally, the weights wij of circle packing metric should be 1, that is, Formula TeX Source $$\gamma _i + \gamma _j = l_{ij},$$for all edges. All neighboring circles are tangent to each other. Since usually |E| > |V |, it is however an overdetermined system. We solve this problem by pursuing the optimal radius configuration in the least-squares sense. The system of Eq. (8) can be written as a matrix-vector multiplication A ¡ = L, where the matrix A has the entries as follows: Formula TeX Source $$A_{rc} = \left\{ \matrix{1\;\,\,\,{\rm{if }}v_c {\rm{ is an incident vertex of the }}r{\rm{th edge}} \hfill \cr {\rm{0}}\,\,\,{\rm{otherwise,}} \hfill \cr} \right.$$and the entries of ¡ and L are the corresponding radii and edge lengths, respectively. Thanks to the sparsity of the matrix A, optimal circle-packing metric can be solved using the Preconditioned Bi-Conjugate Gradient method (PBCG) with great efficiency. The intrinsic geometric diffusion for a given mesh is computed as the following:

Table 1
TABLE 1 Analogy between smooth Surface and Triangular Mesh
Figure

The Ricci flow is a negative gradient flow of the energy: Formula TeX Source $$E_{Ricci} = \sum\limits_i {\int {K_i du_i }.}$$where Ki is the Gaussian curvature at vi, induced by the current metric {ui}. It is ensured that the Ricci flow converges to the global minimum of ERicci. For interested readers, more details on the discretization and numerical issues can be found in [2]. As indicated by Alg. 1 again, the Ricci flow only manipulates the intrinsic geometry (the inner metric) without any further reference to the ambient space. Consequently, vertex positions are not relevant at this stage. Also note that the metrics on the boundary stay non-deformed, such that all boundary edges retain their original length. This treatment ensures that all strict maxima and minima of Gaussian curvature belong to the original shape. This property is of critical importance for the generation of a scale-space shape representation as detailed in the next section.

SECTION 3

Geometric Scale Space

As demonstrated in [17], [19], [16], [23], [14], scale-space representations have been proven an extremely effective tool in analyzing signals with different levels of details. To define a scale space, a set of scale-space axioms that describe basic properties of the desired representation are first established, which largely narrow the choices for a qualified candidate. Although the axioms have been formulated in a variety of ways, specialized for different applications, one property called causality is uniformly imposed in all scale-space representations, which essentially states that no spurious structures should be generated at a coarse scale in the representation without a "cause" at finer scales. By using intrinsic geometry diffusion, a novel geometric scale space can be consequently constructed, which satisfies the causality criterion. We are aware of a similar approach proposed in [8]. In sharp contrast, we analyze a scale space directly built upon the intrinsic surface geometry, which generally preserves more information of the original shape and avoids the complex topological surgery to slice the surface open to a disk.

In this framework, we consider a surface as a Riemannian manifold. The scale-space representation of the surface geometry has been formulated as a family of diffused metrics, parameterized by t in the last section. As 2D images are presented by pixel intensities, the intrinsic geometry of surfaces can be presented by the pointwise Gaussian curvatures. Given a surface S, let S := S(t) evolve with the Ricci flow. By Eq. (3), the Gaussian curvatures will diffuse in the same manner as the image intensity does in images. In practice, our proposed intrinsic geometry scale space is represented as a time-varying Gaussian function K(V t) defined on the vertex set V of a triangular mesh τ=(V E F), with K(vi, 0) being the initial discrete curvature at viV and t ∊ [0, α) the time or scale parameter. More specifically, the IGSS is retained by a n × | V | matrix Formula TeX Source $$IGSS(T) = \left({\matrix{{K_{v_0 }^{t_0 } } {K_{v_0 }^{t_1 } } \ldots {K_{v_0 }^{t_{n - 1} } }\cr {K_{v_1 }^{t_0 } } {K_{v_1 }^{t_1 } } \ldots {K_{v1}^{t_{n - 1} } } \cr \vdots \vdots \ddots \vdots \cr {K_{v_m }^{t_0 } } {K_{v_m }^{t_1 } } \ldots {K_{v_m }^{t_{n - 1} } } \cr } } \right)$$ Each column of the matrix stands for the surface geometry at scale t. In the following, we will show the detailed properties of the IGSS, as well as the automatic feature detection and scale selection scheme for the 3D shape analysis.

3.1 Geometric Scale Space Properties

Causality For 3D objects, recent research in visual saliency suggests that features usually appear as curvature variance [15], [36]. For example, a sphere or a plane produces little visual attractiveness as it has invariant curvature across the surface. For this reason, we identify geometric features as diffused curvature extrema in the IGSS. The causality criterion can be established by requiring that all minima and maxima of the Gaussian curvature function belong to the original shape within the scale space . Based on Eq. (3), it can be formally proved that all the extrema of the solution in space and time arise in its initial condition, i.e., the original shape or, on the boundaries. Because we further suppress the diffusion from the surface boundary in Alg. 1, which is called an adiabatic boundary condition, the IGSS bears all its extrema on the original surface. Thus, it satisfies the causality criterion. A rigorous proof is given in the Appendix.

Scale Invariance In the spirit of discrete differential geometry, the curvatures are defined at vertices, edges, and faces. Particularly with the Euclidean background geometry (assumed throughout this paper) [11], the edges and faces have no curvature. This setting lets the IGSS inherently invariant to the similarity transformation of the objects. This property can also be easily judged from the computation of the discrete curvature in Eq. (5), where only the angles matter. Thus, if a feature is assumed in the IGSS of a certain shape, at scale level t0, then under a rescaling of the shape with a factor s, the corresponding extrema for the rescaled shape will be transferred to scale t0 + f(s), where f(s) is a monotonically increasing function of s. Note that the Ricci flow is a nonlinear geometric flow, as the underlying metric of the surface is being deformed simultaneously. As a consequence, the scale space established through the Ricci flow is also nonlinear. Deriving a closed form of f(s) is non-trivial.

Parallelism Because of the variational solver, diffusion is computationally expensive in general. However, the parallel structure of the algorithm gives rise to a means to speed up the process. First, partition the vertex set V into a number of subset V0 V1, … Vn–1, such that Formula and each vertex is at least an interior vertex in one subset. Therefore, each piece can be computed separately and alternatively. Because the Ricci flow energy is convex, the alternating optimization converges to the global minima, which is equivalent to the sequential execution.

Flexibility Thus far, we assume that the geometry will diffuse to a flat metric, i.e Ki = 0. In case that a non-trivial final metric is desired to guide the diffusion, a per-vertex target curvature distribution Formula can be specified across the vertices, as long as the Gauss-Bonnet condition is satisfied. Accordingly, the diffusion equation (Eq. 4) is modified to Formula TeX Source $${{du(t)} \over {dt}} = \hat K_u - K_u .$$ It warrants that the geometry converges to the specified geometry eventually.

3.2 Scale-Aware Features in Geometric Scale Space

The notion of scale is of fundamental importance when processing heterogeneous, noisy geometry using automatic methods. To allow for a finer analysis, scale dependent feature detection and appropriate feature scale selection is crucial. We complement the IGSS with a mechanism which can automatically detect salient, robust features and determine their local scales, upon which local shape descriptors can be computed for many purposes.

3.2.1 Scale Dependent Feature Detection

A scale space representation by itself contains no explicit information about what structures in the data should be regarded as significant or what scales are appropriate for treating them. Lindeberg [18] attempted to address this problem in the image domain and suggested that local extrema over scales of normalized differential entities may correspond to structures of interests. Essentially, it is a scale-structure matching process. Local maxima of operator responses are assumed at the corresponding scales of the structures of interest. Because the propagation of scale measurement is performed in space via the diffusion equation, scale information has been incorporated for scale-dependent feature detection when performing local evaluation at single points in the scale-space representation.

Once shapes are represented at multiple scales, meaningful features can be extracted in a scale-invariant manner and adaptive to the surface geometry. For 2D images, the Laplacian normalized with the scale parameter t has proved to be a more stable feature detector, compared to a range of other possible candidates, such as the gradient, Hessian, or Harris corner function [18], [19], [20]. We therefore extend it to the surface geometry. Let Kt be the geometric representation at scale t in the IGSS. The scale-normalized Laplacian operator is defined as Formula TeX Source $$\Delta _{norm} K^t = t \cdot \Delta _{g(t)} K^t .$$ To detect scale-dependent features Δnorm Kt is required to be the local maxima/minima in the IGSS with respect to space and scale simultaneously. Given a triangular mesh, the discrete Laplacian at vi can be computed by the well-known cot-Laplace operator: Formula TeX Source $$\Delta K_{v_i }^t = {1 \over 2}\sum\limits_{v_j \in N_1 (v_i)} {(\cot \alpha _{ij} + \cot \beta _{ij})(K_{v_i }^t - K_{v_j }^t)}$$where N1(vi) denotes the 1-ring neighbors of vi αij and βij are the two angles opposite to edge eij in the two triangles sharing eij. Consequently, we obtain Formula TeX Source $$\Delta _{norm} IGSS = \left({\matrix{{t_0 \Delta K_{v_0 }^{t_0 } } {t_1 \Delta K_{v_0 }^{t_1 } } \ldots {t_{n - 1} \Delta K_{v_0 }^{t_{n - 1} } } \cr {t_0 \Delta K_{v_1 }^{t_0 } } {t_1 \Delta K_{v_1 }^{t_1 } } \ldots {t_{n - 1} \Delta K_{v_1 }^{t_{n - 1} } } \cr \vdots \vdots \ddots \vdots \cr {t_0 \Delta K_{v_m }^{t_0 } } {t_1 \Delta K_{v_m }^{t_1 } } \ldots {t_{n - 1} \Delta K_{v_m }^{t_{n - 1} } }\cr} } \right)$$ Feature points are identified as local minima/maxima of the normalized Laplacian of the IGSS representation across scale t. Specifically, it is done by comparing each vertex in the mesh structure to its 1-ring neighbors at the same scale t and also itself as well as its 1-ring neighbors at the neighboring scales t − 1 and t + 1, If the vertex bears the maximum or minimum among all compared vertices, it is selected as a feature point. Fig. 1(a) illustrates the neighborhood of a feature point detected in the geometric scale space. The red dot denotes the key-point (extremum), while the other surrounding green dots denote its neighborhood in space and over scales.

3.2.2 Geodesic Scale Computing

One merit of scale space is that features can be extracted with respect to their inherent scales. Feature scales are critical in a scale-space setup, since all subsequent local computations are conducted on this feature-adaptive support. In the scale spaces formulated via Gaussian smoothing [19], [15], [36], the scale of a detected keypoint is empirically defined as certain multiple of the standard deviation of the current Gaussian kernel. The support region is centered at the keypoint. A quantitative connection between spatial scale σ and the diffusing time t exists in the Euclidean space as Formula given by the general solution of a diffusion equation. However, this equality does not hold in general for an intrinsic geometry diffusion, since the metric is deformed. We follow the intuition of heat diffusion, defining the feature scales on the manifolds through front propagation. Suppose S(t) is the diffused surface at time t, and a point Formula is detected as a feature point at S(t). The scale Formula is delineated by a center-surrounded geodesic neighborhood u(v) of Formula under metric g(0), which is formulated as Formula TeX Source $${\cal U}(v,\,\,t) = \{ x|dist(x,\;\tilde v) < \sqrt {2t},\;x \in S(t)\},$$

Figure 1
Fig. 1. Feature extraction in the geometric scale space. (a) shows the neighborhood setting of a keypoint; (b) visualizes the geodesic scale associated with the keypoint.

where dist Formula denotes the geodesic distance between points Formula and x, estimated under the original metric. In practice, such neighborhoods can be estimated by performing isotropic front propagation on the triangle mesh, originating from each keypoint. Let C(s) be a level set curve of dist on (x Formula) hence centered at Formula and s the arc length parameter. The propagating velocity of the front is given by Formula TeX Source $$a\left| {p_i - p_j } \right|^\alpha + b\left| {s(p_i)} \right. - s(p_j)\left| {^\beta } \right..$$where Formula is the exterior unit vector normal to the curve C(s) at s in the tangential plane of s, and c is chosen to be the 1/10 of the average edge length Formula is then given as the region bounded by C(s) at time t. Propagation is stopped once Formula reaches the size as defined by the corresponding scale value as in Eq. (16). Fig. 1(b) shows the geodesic scale of a keypoint defined on the surface. The process of front propagation is visualized by a color map from red (t = 0) to blue (t = tn) as time elapses.

SECTION 4

Applications and Experimental Results

The IGSS provides a unified methodology for the automatic detection of scale-dependent salient geometric feature on the surface. When integrated with a 3D local shape descriptor, it can also serve as a feature-based 3D shape matching framework. In the order of saliency identification, feature extraction and shape matching, we evaluate the effectiveness of IGSS in the following applications, respectively.

4.1 Saliency-based Surface Simplification

For the interactive visualization of large-scale 3D models, surface simplification is often needed to downgrade the original model to a simplified version, in order to accommodate the graphics hardwares. The essential goal of surface simplification is to preserve the visual appearance of the original model as far as possible with a given triangle budget. Conventionally, an error metric is first defined based on some geometric primitives, such as curvatures [4]. Then, such metric is used to guide the simplification process, e.g., through iterative edge contraction. These methods in general incline to preserve regions of high curvatures while, on the other hand, decimate more intensively in somewhat flat regions. However, it should be noted that the attribute of curvature is only capable of conveying the local geometry, which usually does not contain sufficient information to indicate perceptually interesting regions. By taking scale-dependent features into consideration during the mesh simplification, and guiding the process in a feature/scale-aware manner, more visually pleasing results can be achieved [15]. Since the IGSS's center-surrounded mechanism can also serve as the visual front-end in a vision system for 3D shape analysis, we shall present a competitive alternative to the previous methods in capturing visually salient regions of 3D models [15] and demonstrate its leverage in surface simplification.

Once a feature point (vertex v is detected as the local extremum in the IGSS, it is natural to define the magnitude of the scale-normalized Laplacian (LδKv) at v as the measure of feature strength. In addition, the associated scale information defines a support region for the feature centered at v. We assign each vertex a saliency value φ within its scale-dependent neighborhood, based on the strength of the feature it belongs to: Formula TeX Source $$\phi (x,\,v) = |t\Delta K_v |,\quad x \in {\cal U}(v,t),$$where u(v,t) denotes the geodesic local support of v, determined by scale t. When multiple features are close to each other on the surface, the respective support regions may have overlaps. In such a case, the final saliency of a point is defined as the maximum among all saliency values attributed to it: Formula TeX Source $$\tilde \phi (x) = \mathop {\max }\limits_{k:x \in {\cal U}(v_k,\;t)} \phi (x,\;v_k).$$Therefore, a saliency map is established on the mesh through the IGSS.

For the purpose of comparison, we apply our saliency map to the same basic simplification framework as in [15]. In this framework, simplification algorithm is based on the iterative contraction of vertex pairs Formula. An error metric Q is defined at each vertex. To select a pair to contract, a cost is associated with each pair as Qij = Q(vi) + Q(vj), and at each iteration, the pair of least cost is collapsed. More detailed information can be found in [4]. Now suppose S(V) is the saliency map defined on the vertex set V of a triangular mesh. The error metric is modified as Formula TeX Source $$\tilde Q(v) = S(v)Q(v).$$ Accordingly, the cost for contracting vertex pair (vi vj) is computed as Formula. Fig. 2(a) and 2(d) show the saliency maps derived from the IGSS of the Neptune and the Elephant models. Evidently, those visually interesting features, such as the head of Neptune, the spoke of the trident, and the eye of the Elephant, are successfully detected as salient sub-parts of the whole shape, as shown in Fig. 2(a) and Fig. 2(d). Reasonable salient features are successfully detected as expected. Fig. 2(b), 2(c), 2(e), and 2(f) show the results of simplifying the Neptune model and the Elephant model, both with a resolution of 50, 000 faces. Fig. 2(b) gives the result of the Neptune decimated to 1, 000 faces, using the Qslim method [4], whereas in Fig. 2(c), the improved result with the same budget is achieved by using our method. Consistent results is also obtained from the Elephant model, as shown by Fig. 2(e) and Fig. 2(f).

4.2 Scale-dependent Feature Extraction

Recently, the relative scale variability of local geometric structures has received much attention, especially in 3D vision applications [23]. With a rich set of scale-dependent 3D features detected from the IGSS, a number of tasks, such as shape matching [12], 3D face recognition [36], surface registration [37], can greatly benefit from this framework.

Fig. 3 shows the geometric features extracted from three 3D models: the Julius Caesar, the Armadillo and the Buddha. Each feature is visualized by a sphere centered at the keypoint, whose radius is proportional to the feature's associated scale. Because features at small scales (empirically t < 0.005) could be possibly due to the noise, those features have been suppressed. Furthermore, thresholding on the magnitude of the scale-normalized Laplacian constitutes another level of feature selection, when too many features cause visual cluttering. Observe that the features spread across various scales and the scale of a specific feature is consistent with the scale of the underlying surface geometry. Take the Caesar model as an example. While many small features are detected around regions of eyes, nose, and mouth, each part is extracted as one single feature at coarser scales. This phenomenon reflects the hierarchical nature of the scales of detected features.

Figure 2
Fig. 2. Saliency-guided surface simplification. (a) and (d) show the saliency map derived from the IGSS. (b) and (e) are the results obtained by Qslim method [4] with a budget of 1, 000 faces; (c) and (f) show the improved results achieved by using our method, given the same budget.
Figure 3
Fig. 3. Keypoint Detection. (a) Julius Casear; (b) Budda; (c) Armadillo.

We further test the repeatability of feature detection under noises. In this experiment, random noises are injected to the surface of the Caesar model along the normal direction. The noise magnitudes span from 0 to 5%, 7.5%, and 10% of the bounding ball radius, respectively. As shown by Fig. 4, despite that small features may vary in location because of the injected noises, features detected at larger scales are highly consistent across all cases, which are robustly detected at close locations and scales. We also performed comparison experiments among several related works, including mean curvature flow-based scale space processing [36] and geodesic shape vector image diffusion [8]. Average results of the feature point repeatability are shown in Fig. 5. Because the IGSS purely relies on intrinsic properties of shapes, extracted features are generally more robust to the noise than the other two.

Figure 4
Fig. 4. Scale-dependent feature extraction at the presence of noise: (a) 5%; (b) 7.5%; (c) 10%, relative to the bounding ball radius.
Figure 5
Fig. 5. Comparison of feature point repeatability under random noise.

4.3 3D Shape Matching

3D shape matching is a fundamental issue in computer vision and geometry processing [36]. This experiment is performed on a 3D face database which contains 100 3D facial scans from 20 subjects to analyze the performance of our proposed framework. A scale-dependent geodesic fan shape representation [5], [24] is employed as the local shape descriptor. Each subject has 5 different expressions. Our method can correctly match the same subject with different expressions while differentiating different subjects based on the number of matched key-points. Descriptor matching is obtained for a keypoint by comparing the distance from its constructed local descriptor to its closest neighbor with the distance to that of the second-closest neighbor found on the to-be-matched object [8]. Only when the distance to the closest neighbor is much greater than the distance to the second-closest neighbor, we consider they are matched keypoints. This ensures that only distinctive keypoints having prominent similarity are matched. A threshold is given here to judge the distinctiveness. Fig. 6(a) shows the matched same subject with different expressions and Fig. 6(b) shows the differentiation between two different subjects, where the number of matched keypoints is significantly fewer. Fig. 7 shows the statistics of the numbers of matched keypoints among 20 subjects with different expressions. As we can see, the numbers of matched keypoints are significantly higher in the intra-subject matching (same subject different expressions) against those found in the inter-subject case. Note that only matches with high confidence (small matching distance) are selected. The number of matched keypoints is descriptive enough to differentiate models from different subjects and, meanwhile, retrieve distinct expressions of the same person. The experiments indicate that our method is effective for 3D shape matching, recognition and retrieval.

We have also compared with other methods in the face retrieval experiment. The faces with the top numbers of matched points to a query are considered as the retrieved faces. Assuming the class size is Cn, the First Tier shows the percentage of correctly related items within the top Cn of all ranked lists and the Second Tier shows the percentage within the top 2 × Cn. Table. 2 shows the comparison between the IGSS and other methods [31] in terms of three different measures, namely the recognition rate (RR), the first and the second Tier.

Figure 6
Fig. 6. Keypoint matching. (a) shows matched salient feature points between the different expressions of the same subject; (b) shows the few matched feature points between different subjects.
Figure 7
Fig. 7. Comparison of the numbers of matched keypoints between intra-subject and inter-subject matches among 20 subjects under a fixed distinctiveness threshold.
SECTION 5

Conclusion

In this paper, we have presented a novel, formal intrinsic geometric scale space constructed by shape diffusion through the Ricci flow. This multiscale shape representation satisfies a range of axiomatic scale-space properties. It provides a formal means to study the geometric scalability and variability, as well as their effects in graphics and vision applications. In the geometric scale space, we have also proposed a feature-based shape representation based on the computation of scale-aware local shape descriptors within the local support scales where the feature points are detected. Promising results are obtained through examples of salient feature detection, scale-dependent feature extraction, and 3D facial scan matching.

Table 2
TABLE 2 Results of retrieval of faces. Top 4 runs in different methods [31] are compared with IGSS.

Acknowledgments

This work is supported in part by the following research grants: NSF IIS-0915933, IIS-0937586, IIS-0713315, and CNS-0751045, as well as NIH 1R01NS058802-01 and 2R01NS041922-05.

Appendix

Suppose S: D → ℝ3 is a smooth surface embedded in ℝ3 where D denotes the domain 2-manifold; the induced Euclidean metric is g. In the following discussion, we separate the topology and geometry of S by denoting S as a pair (D g). Let T = (a b) be an interval of ℝ representing the range of scale parameter t. The geometric scale space L(D g) is therefore formed by the product L = (D g) × T = {(p g(p t))| pD tT}. Furthermore, let ∂ L represent the boundary of L, and ∂tL, ∂ sL, and ∂bL the top, side, and bottom portions of ∂ L, respectively, given by Formula TeX Source $$\eqalign{ \partial _t L = \{ (p,g(p,t))|p \in D,t = a\} \cr \partial _S L = \{ (p,g(p,t))|p \in \partial D,t \in T\} \cr \partial _b L = \{ (p,g(p,t))|p \in D,t = b\} \cr \partial L = \partial _t L \cup {\rm{ }}\partial _S L \cup {\rm{ }}\partial _b L. \cr}$$ We shall prove the causality of geometric scale space by showing that the Gaussian curvature function Kg(t) : D → ℝ (K is defined as the geodesic curvature on ∂ D), as well as all its spatial derivatives ∇S K, reach the maximum and the minimum on the bottom ∂bL of L.

Lemma 1 (The Maximum Principle of Intrinsic Geometric Diffusion Consider a smooth function f: L → ℝ If f satisfies Formula TeX Source $$F = f_t - \Delta _{g(t)} f \le 0,$$where δg(t) is the Laplace-Beltrami operator under metric g(t then f obeys the maximum principle: Formula TeX Source $$\mathop {\max }\limits_L f = \mathop {\max }\limits_{\partial _b L} f.$$Proof. By hypothesis L is compact and closed, hence f has a maximum in it. We denote the maximum by pmax = (p t). First consider the case when pL\ ∂ L. Let m = fλ(ta), for any λ > 0. Provided that f satisfies the weak inequality Eq. (22 m satisfies the strict inequality Formula TeX Source $$M = m_t - \Delta _{g(t)} m < 0.$$ In the local neighborhood Tp × ℝ of p where Tp represents the tangential space of p m(r) can be expanded by the Taylor series as Formula TeX Source $$m(r) = m(p) + \nabla m^T (r - p) + {1 \over 2}(r - p)^T {\cal H}(m)(r - p) + {\cal O}(||r - p||^3) \le m(p),$$ where Formula is the Hessian matrix of m. Since p is a point where m has a maximum, the gradient ▽m = 0. Consequently, the quadratic term of Eq. (25) has to be non-positive to hold the inequality, i.e Formula. It also implies that the Hessian matrix Formula is negative semidefinite and the diagonal entries are either equal to zero or negative. Notice that the Laplace-Beltrami operator is the trace of the Hessian Formula with respect to the metric, restricted to Tp × ℝ Thus Δg(t) m ≤ 0. Recall that mt is the t-component of ∇m. Since mt = 0 at p, then Formula TeX Source $$m_t - \Delta _{g(g)} m \ge 0,$$ contradicting Eq. (24). Thus, there is no local maxima of m in the interior of L.

The same argument is also valid for ∂sL as the Laplace-Beltrami operator is performed on the boundary curves. Similarly, if p ∊ ∂tL, we have mt ≥ 0 and ▽sm=0, which is an even stronger condition for deriving the contradiction than in the argument above. Thus m does not have local maxima in ∂sL or ∂tL either. It ensures that m obeys the maximum principle: maxLm = maxbL m.

Since f = m + λ(ta) ≤ m + λ(ba) on L, we can see that Formula TeX Source $$\mathop {\max }\limits_L f \le \mathop {\max }\limits_L (m + \lambda (b - a)) = \mathop {\max }\limits_{\partial _b L} (m + \lambda)(b - a)) \le \mathop {\max }\limits_{\partial _b L} (f + \lambda (b - a)).$$ Letting λ → 0, we obtain Formula

Now we show that the Gaussian curvature function on surface (D g) and its derivatives obey both the maximum and minimum principle.

Theorem 1 (The Causality of Intrinsic Geometric Scale Space The Gaussian curvature function Kg(t) : D → ℝ induced by the initial geometry and the Ricci flow, and all derivatives of Kg(t) in space satisfy both the maximum and minimum principle: Formula TeX Source $$\mathop {\max }\limits_L f = \mathop {\max }\limits_{\partial _b L} f\quad and\quad \mathop {\min }\limits_L f = \mathop {\min }\limits_{\partial _b L} f.$$Proof. First consider the maximum principle. By the governing equation of the surface Ricci flow, Eq. (3), the Gaussian curvature function K satisfies Eq. (22) since Formula TeX Source $${{\partial K} \over {\partial t}} - \Delta _{g(t)} K = 0,$$ thus max maxL K = maxb L K. Furthermore, let Formula be a certain differential operator in space. Apply Formula to Eq. (29). Due to the commutative property of differentiation, we have Formula TeX Source $$\displaylines{{{\partial {\cal D}K} \over {\partial t}} - \Delta _{g(t)} {\cal D}K = {\cal D}({{\partial K} \over {\partial t}}) - {\cal D}(\Delta _{g(t)} K) \cr = {\cal D}({{\partial K} \over {\partial t}} - \Delta _{g(t)} K) = {\cal D}(0) = 0. \cr}$$ Again, since all spatial derivatives of K satisfies Eq. (22), they also obey the maximum principle.

Notice that if a function f satisfies Eq. (22), so does its negative h = − f. The argument above also holds for h, such that Formula TeX Source $$\mathop {\max }\limits_L h = \mathop {\max }\limits_{\partial _b L} h\quad \Rightarrow \quad \mathop {\min }\limits_L f = \mathop {\min }\limits_{\partial _b L} f.$$It indicates the Gaussian curvature function and all the spatial derivatives obey the minimum principle as well.

Theorem 1 shows that all maxima and minima of the Gaussian curvature belong to the original 3D shapes. Therefore, our proposed scale-space representation has the axiomatic property of causality.

Footnotes

G. Zou, J. Hua, Z. Lai, and M. Dong are with Wayne State University, E-mail: gyzou@cs.wayne.edu| jinghuagyzou@cs.wayne.edu| kevinlaijinghuagyzou@cs.wayne.edu| mdong@cs.wayne.edu.

X. Gu is with Stony Brook University, E-mail: gu@cs.sunysb.edu.

Correspondence to Prof. Jing Hua.

Manuscript received 31 March 2009; accepted 27 July 2009; posted online 11 October 2009; mailed on 5 October 2009. For information on obtaining reprints of this article, please send email to: tvcg@computer.org.

References

1. Speeded-up robust features (surf).

H. Bay, A. Ess, T. Tuytelaars and L. Van Gool

Comput. Vis. Image Underst., 110 (3): 346–359, 2008.

2. Combinatorial ricci flows on surfaces.

B. Chow and F. Luo

J. Differential Geom., 63 (1): 97–129, 2003.

3. Implicit fairing of irregular meshes using diffusion and curvature flow.

M. Desbrun, M. Meyer, P. Schröder and A.H. Barr

In Proceedings of SIGGRAPH '99, pages 317–324, 1999.

4. Surface simplification using quadric error metrics.

M. Garland and P.S. Heckbert

In Proceedings of SIGGRAPH '97, pages 209–216, 1997.

5. Curvature maps for local shape comparison.

T. Gatzke, C. Grimm, M. Garland and S. Zelinka

In Proceedings of the Int'l Conf. on Shape Modeling and Applications 2005, pages 246–255, 2005.

6. Ricci flow for 3D shape analysis.

X. Gu, S. Wang, J. Kim, Y. Zeng, Y. Wang, H. Qin and D. Samaras

In Proceedings of ICCV '07, pages 1–8, 2007-10.

7. The ricci flow on surfaces.

R.S. Hamilton

Math. and General Relativity, Contemporary Math., Am. Math. Soc., 71, 1988.

8. Geodesic distance-weighted shape vector image diffusion.

J. Hua, Z. Lai, M. Dong, X. Gu and H. Qin

IEEE Trans. on Visual. and Comput. Graph., 14 (6): 1643–1650, 2008.

9. Representations based on zero-crossings in scale-space.

R.A. Hummel

In Readings in computer vision: issues, problems, principles, and paradigms, pages 753–758, 1987.

10. Discrete mean curvature flow.

A. Imiya and U. Eckhardt

In Proceedings of the Second International Conference on Scale-Space Theories in Computer Vision, pages 477–482, 1999.

11. Discrete surface ricci flow.

M. Jin, J. Kim, F. Luo and X. Gu

IEEE Trans. on Vis. and Comput. Graph., 14 (5): 1030–1043, 2008.

12. Using spin images for efficient object recognition in cluttered 3D scenes.

A.E. Johnson and M. Hebert

IEEE Trans. Pattern Anal. Mach. Intell., 21 (5): 433–449, 1999.

13. Intrinsic scale space for images on surfaces: The geodesic curvature flow.

R. Kimmel

Graph. Models Image Process., 59 (5): 365–372, 1997.

14. Scale-space processing of point-sampled geometry for efficient 3D object segmentation.

H. Laga, H. Takahashi and M. Nakajima

In Proc. the 2004 Int'l Conf. on Cyberworlds, pages 377–383, 2004.

15. Mesh saliency.

C.H. Lee, A. Varshney and D.W. Jacobs

ACM Trans. Graph., 24 (3): 659–666, 2005.

16. Multi-scale features for approximate alignment of point-based surfaces.

X. Li and I. Guskov

In Proceedings of the third Eurographics symp. on Geometry processing, pages 217–228, 2005.

17. Scale-Space Theory in Computer Vision

T. Lindeberg

Kluwer Academic Publishers, 1993.

18. Feature detection with automatic scale selection.

T. Lindeberg

Int. J. Comput Vision, 30 (2): 79–116, 1998.

19. Distinctive image features from scale-invariant keypoints.

D.G. Lowe

Int. J. Comput Vision, 60 (2): 91–110, 2004.

20. An affine invariant interest point detector.

K. Mikolajczyk and C. Schmid

In Proceedings of the 7th European Conference on Computer Vision, pages 128–142, 2002.

21. A performance evaluation of local descriptors.

K. Mikolajczyk and C. Schmid

IEEE Trans. Pattern Anal. Mach. Intell., 27 (10): 1615–1630, 2005.

22. Multi-scale free-form 3D object recognition using 3D models.

F. Mokhtarian, N. Khalili and P. Yuen

Image and Vision Computing, 19 (5): 271– 281, 2001.

23. Scale-dependent 3D geometric features.

J. Novatnack and K. Nishino

In Proceedings of ICCV '07, pages 1–8, 2007.

24. Scale-dependent/invariant local 3D shape descriptors for fully automatic registration of multiple sets of range images.

J. Novatnack and K. Nishino

In Proceedings of the 10th European Conf. on Computer Vision, pages 440–453, 2008.

25. Multi-scale feature extraction on point-sampled surfaces.

M. Pauly, R. Keiser and M. Gross

Comput. Graph. Forum, 22 (3): 281–289, 2003.

26. Point-based multiscale surface representation.

M. Pauly, L.P. Kobbelt and M. Gross

ACM Trans. Graph., 25 (2): 177–193, 2006.

27. Scale-space and edge detection using anisotropic diffusion.

P. Perona and J. Malik

IEEE Trans. Pattern Anal. Mach. Intell., 12 (7): 629–639, 1990.

28. The convergence of circle packings to the riemann mapping.

B. Rodin and D. Sullivan

J. Differential Geom, 26: 349–360, 1987.

29. Conformal equivalence of triangle meshes.

B. Springborn, P. Schröder and U. Pinkall

In Proceedings of SIGGRAPH '08, pages 1–11, 2008.

30. Approximation of conformal structures via circle packing.

K. Stephenson

In Proceedings of Computational Methods and Function Theory 1997, pages 551–582, 1999.

31. SHape REtrieval Contest 2008: 3D Face Scans.

F. B.t er Haar, M. Daoudi and R.C. Veltkamp

In Proceedings of the Shape Modeling International 2008, pages 225–226, 2008.

32. The geometry and topology of 3-manifolds.

W. Thurston

In Princeton University Lecture Notes, 1976.

33. Scale-space filtering.

A.P. Witkin

In 8th Int'l Joint Conf. Artificial Intelligence, volume 2, pages 1019–1022, 1983.

34. Surface fairing and featuring by mean curvature motions.

G. Xu

J. Comput Appl. Math., 163 (1): 295–309, 2004.

35. Triangular surface mesh fairing via gaussian curvature flow.

H. Zhao and G. Xu

J. Comput Appl. Math., 195 (1): 300–311, 2006.

36. Surface matching with salient keypoints in geodesic scale space.

G. Zou, J. Hua, M. Dong and H. Qin

Comput. Animat. Virtual Worlds, 19 (3-4): 399–410, 2008.

37. Non-rigid surface registration using spherical thin-plate splines.

G. Zou, J. Hua and O. Muzik

In Proceedings of MICCAI '07, pages 367–374, 2007.

Authors

No Photo Available

Guangyu Zou

No Bio Available
No Photo Available

Jing Hua

Member, IEEE

No Bio Available
No Photo Available

Zhaoqiang Lai

No Bio Available
No Photo Available

Xianfeng Gu

No Bio Available
No Photo Available

Ming Dong

No Bio Available

Cited By

No Citations Available

Keywords

IEEE Keywords

No Keywords Available

More Keywords

No Keywords Available

Corrections

No Corrections

Media

No Content Available

Indexed by Inspec

© Copyright 2011 IEEE – All Rights Reserved