We present a new approach to sampling features in volumetric data with dynamic particle systems. Particle systems appear most often in computer graphics and visualization for sampling or meshing implicit surfaces from geometric modeling or pre-computed image segmentation. One reason for the success of particle systems is that they cleanly separate the task of computing an optimal distribution of vertices on a feature, from the task of connecting the vertices into a polygonal mesh (whereas Marching Cubes [44], for example, does both at the same time). Calculating vertex locations is done with some combination of energy minimization and constraint satisfaction, both of which can be implemented in a straight-forward manner at the level of individual particles, since the required information is limited to a small neighborhood of particles and data values. Particles also enjoy the property of existing in a mathematically continuous representation of the image, so the underlying image resolution and sampling grid have no undue influence. Our work leverages these properties while it extends particle systems into new feature types and image domains.

Effective computational analysis of medical imaging data often requires geometric models of anatomic features. Challenges to feature localization and extraction arise from image noise and limited resolution, as well as from the properties of features themselves, such as branching or variable geometry, or having a wide range of characteristic sizes *Scale-space* is a computer vision framework for robust feature extraction, in which an *n*-D image is considered as a (*n* + 1)-D stack of images at successive blurring levels, so that large and small features can be detected with equal ease. While theory and methods for scale-space of grayscale, two-dimensional images are well-studied, the promise of scale-space analysis has yet to be realized in practical tools for three-dimensional imaging. Scale-space analysis of non-scalar data, such as diffusion tensor imaging, is largely unexplored.

We propose that particle systems can play a more fundamental role in biomedical visualization and analysis, by sampling complex anatomic features in *unsegmented* data. We focus on ridge and valley features (collectively *creases)* that can approximate the skeletal cores of objects, rather than the closed isocontours and material boundaries of previous particle systems. Creases are either codimension-one surfaces or codimension-two curves, defined as local extrema with respect to motion along Hessian eigenvectors. Crease surfaces are in general non-orientable, and often appear as multiple disjoint surface patches rather than as closed boundaries. A major challenge associated with using crease features to represent objects in images is that objects naturally occur at a range of sizes relative to the pixel resolution. Scale-space particles locate and sample creases throughout their continuous extent in space and scale. The scale at which a crease appears strongest relates to the characteristic size of the underlying object.

Particle systems are particularly well-suited to crease feature sampling and visualization in scale-space. The mathematical definition of creases directly maps to the rules that individual particles obey to satisfy the constraint of lying within a crease. The implementation of pair-wise particle dynamics, which ultimately drives the system, is the same regardless of feature codimension (surfaces versus lines), and image domain dimension (three or four). In real-world applications, we often don't know where features are, or in how many disjoint pieces they appear; this corresponds to the nature of particles and their lack of explicit topology and connectivity. The point-based representation allows us to simply initialize the system with particles densely sampling the entire image domain, and later reduce the number of particles according to their density on the detected features. At this initial stage of work, however, we are refraining from tackling the task of *meshing* the particle system solution. Subsequent research will investigate the computational geometric considerations for reliably connecting the final particle locations into polygonal feature models.

Our contributions stem from how we design, implement, and apply the combination of particle systems and scale space. At the lowest level, we introduce in Section 3.2 a novel Hermite spline approach for efficiently interpolating through image scales to create a continuous, four-dimensional scale-space. Generalizing the implicit surface constraint previously used for particles, Section 3.3 describes constraints that keep particles within ridges and valleys. We introduce in Section 3.4 inter-particle energy functions that allow particles to either repel or attract along scale, so that the features can either be broadly sampled through scale-space, or be localized at the particular scale that maximizes feature strength. Another novel aspect of our implementation (Sect. 3.5) is that population control (the adding and deleting of particles) is formulated in terms of the same energy minimization that drives the particles towards uniform sampling. We use glyph-based visualizations (Sect. 3.5) to inspect the local properties and over-all configuration of the particle system. Our results (Sect. 4) include visualizations of scale-space particles sampling the branching airways in lung CT, and white matter features in diffusion tensor MRI.

There are three research areas our work draws upon: scale-space feature extraction (Sect. 2.1), particle systems (Sect. 2.2), and Diffusion Tensor Imaging analysis (Sect. 2.3). Connections to previous work establishing the biomedical utility of crease lines and crease surfaces are drawn in Sections 2.1 and 2.3, respectively.

### 2.1 Scale-Space Analysis and Crease Lines

The concept of scale and its importance for computer vision led to scale-space theory, which embeds a signal *f*(*x*) in a one-parameter family of functions *L*(*x; t*), with diffusion time *t* inducing Gaussian blurring with standard deviation [69], [35], [66], Florack *et al.* show how principles of linearity, scale-invariance, and well-posed differentiation also imply the Gaussian kernel, independent of a diffusion process [21]. Koenderink notes that significant image features exist at a continuous scale *range* and conceives of image understanding as happening at all scales simultaneously, rather than at a discrete set of blurring levels [35], A number of scale-space feature-extraction methods have been developed from these ideas. Gauch and Pizer propose multi-resolution analysis for ridge and valley lines by projecting the Hessian at different scales into the level curve tangents, while pointing out the difficulties caused by working on a discrete grid [24], Eberly presents a general description for ridge detection in *n*-D signals without focusing on the scale-dependent nature of the problem [17], an extension that was theoretically described by Miller and Furst [49].

A prominent application of ridge and valley analysis is the extraction of tubular structures, like airways and blood vessels, from medical imaging [2], [22], [3], Lindeberg studies the problem of finding the optimal scale of features (including ridges) by maximizing measurements of feature strength along scale [41]. There is, however, no explicit enforcement of the spatial continuity of feature scale, which we address with a novel scale-attractive inter-particle energy function (Sect. 3.4). An initial application of our work is in airway analysis in lung CT, a modality commonly used to assess various pulmonary conditions [59]. For some diseases, such as chronic obstructive pulmonary disease (COPD), the narrowest airways (near the limits of imaging resolution) are the most clinically significant [28]. This significance creates a need to accurately sample features at the smallest scales, which we accomplish with smooth spatial-domain reconstruction (Sect. 3.2.2), while our scale interpolation (Sect. 3.2.1) captures the connected branching structures continuously across scale.

In the visualization community, there has been some work in exploring scale-space for vector flow-field visualization and feature extraction. Bauer and Peikert use scale-space concepts to recover vortex lines via the parallel vector operator [54] in smoothed volumes [6], Klein and Ertl track vector field critical points through discretely sampled scales, to better filter out noise effects [34], In the image processing community, non-linear filtering of tensor fields has been studied [13], [65], [9], which is conceptually related to scale-space by its mathematical connection to earlier work by Perona and Malik on nonlinear scale-space [55], but there has been relatively little study of the scale-space features in tensor fields. Florack and Astola argue that a scale-space treatment of diffusion tensor fields should respect commutativity of blurring and tensor inversion, and describe the mathematical components of the resulting non-linear scale-space [19]. We contribute to this previous work by explicitly locating and uniformly sampling the geometry of image features in tensor fields, across their full extent in space and scale, using particle systems.

### 2.2 Particle Systems

Particle systems have been studied in computer graphics for various applications; the application most closely related to ours is surface representation. Szeliski and Tonnesen model synthetic surfaces using systems of oriented particles with non-rotationally symmetric energy profiles that self-assemble into surfaces [62], Witkin and Heckbert used a system of particles with rotationally symmetric energy profiles to interactively sample and sculpt implicit surfaces [70], [27], Continued refinements have enabled further applications to shape modeling and other applications (*e.g.* see [61] and references therein).

Other studies outside computer graphics use particle systems in a more data-driven way for scientific or biomedical applications, including interactive scientific visualization [52], anisotropic mesh generation [8], [72], feature-aware mesh smoothing [30], visualization of Smoothed Particle Hydrodynamics [42], and illustrative volume visualization [10]. For medical image analysis, Cates *et al.* develop entropy-based particle systems that simultaneously sample surfaces across multiple volumes, efficiently determining surface correspondences and modes of shape variation [12], [11]. Isosurface sampling is a prominent application of data-driven particle systems [57], [15]. This has been studied in detail by Meyer *et al.,* with curvature-dependent sampling [48] and sampling of isosurfaces in high-order finite element grids [46], Later work connects particles into meshes representing segmented object surfaces [45] and the topology of multi-material volumes [47]. We follow the example of Meyer *et al.* in first determining particle motion, interaction, and population control, while leaving the separate (and significant) computational geometry task of computing vertex connectivity to later work.

A limited amount of previous work shares our approach of using particles to accomplish feature sampling in *unsegmented* data. Szeliski *et al.* direct their oriented particles [62] with an edge detection energy term to form surface models of segmented and unsegmented volume objects [63], More recently, Jalba *et al.* attach fixed electrostatic charges to the edge pixels or voxels in an image, which then attract mutually repelling and mobile particles that converge in tracing out region boundaries [29]. Other previous work, on active contours, creates long-range attractive forces based on gradient vector flows to ensure insensitivity to initialization [71], A virtue of particles demonstrated by Jalba *at al.* [29] is that they can be initialized at every image pixel, ensuring that no significant edge features are missed, which side-steps the problem of sensitivity to initial conditions. However, the magnitudes of the data-driven force (for feature localization) and the inter-particle force (for uniform sampling) must be calibrated so that edges attract particles more strongly than particles repel each other.

Relative to this previous work, our contributions are (1) employing spatial *constraints,* rather than forces or potentials, to maintain particles within features, and (2) moving particles smoothly through scale to sample features in continuous four-dimensional scale-space. With its emphasis on feature detection and localization, our use of particles is conceptually related to active contours and deformable models that use energy-minimizing parameterized curves or surfaces to capture image features [31], Our particles, however, are not connected in any pre-determined topology, and are not described by any particular model of feature geometry, so they can sample codimension-one surfaces as easily as codimension-two lines.

### 2.3 Diffusion Anisotropy Analysis and Crease Surfaces

Diffusion tensor magnetic resonance imaging (DTI) uses a tensor model of diffusion-weighted MRI to describe directional tissue structure [4], The coherent organization of axons in nervous tissue contributes to the anisotropic diffusion of water within it, enabling research on white matter organization in health and disease [37], Much interest in DTI stems from using integrals of the principal eigenvector of the diffusion tensor (termed *tractography)* to approximate axonal connectivity between different brain areas [14], [5], On the other hand, much of the neuroscientific research using DTI focuses on scalar-valued tensor invariants such *as, fractional anisotropy* (FA), which are local measures of microstructural organization [56], [7], Many applications of DTI amount to correlating statistically significant differences in FA with various neurological or psychiatric conditions. For example, Kubicki *et al.* review such methods in schizophrenia [36], Detecting local changes in tissue properties (through measures like FA) is a fundamental neuroimage analysis task, separate from but complementary to connectivity measurements from the tractography methods typically studied in the visualization community.

To improve the statistical validity of voxel-based group studies of FA variations, Smith *et al.* developed Tract-Based Spatial Statistics (TBSS) [60], TBSS rasterizes FA ridge surfaces in a high-resolution, group mean, tensor image to form a white matter skeleton for statistical tests. Other work has similarly analyzed the differential structure of FA to measure anatomic features in DTI. Goodlett *et al.* use an FA ridge surface strength metric to improve registration in group studies [26], analogous to previous registration work for scalar images [43]. Kindlmann *et al.* use a modification of Marching Cubes to extract polygonal models of FA ridge and valley surfaces [33], Tric-oche *et al.* study general invariant crease lines in tensor fields and detect some white matter structures as FA ridge lines [64], Schultz *et al.* present a new approach to extracting crease surfaces that improves on previous voxel-based methods [23], [33] in speed and topological correctness, which they apply to DTI anisotropy measures [58],

Our work relates to these previous results in two ways. First, along with lung airway segmentation (Sect. 2.1), extensions to TBSS [60] are a driving application. We plan to expand the target of the TBSS white matter skeletonization from the smooth group mean image to single-subject images. Initial experiments with this, as well as experience with anisotropy crease surfaces [33] and lines [64], have highlighted the problem with picking a single *scale* to measure derivatives: features at significantly smaller or larger scales are poorly captured or not detected at all. Second and more generally, we suspect that the essential scale-dependence of derivative measurements [20], combined with the combinatorial complexity of extending Marching Cubes-type approaches to four-dimensional scale-space lattices [23], [18], may have limited the adoption of previous crease surface extraction methods, as compared to the ubiquity of isosurfaces from Marching Cubes itself [44], We hope our initial results with scale-space particles can expand the exploration and application of scale-space creases in real-world three-dimensional datasets.

### 3.1 Methods overview

The particles in our system are moved, added, and deleted to minimize their collective energy. There are two energy sources: *particle-image* and *inter-particle.* A combination of constraints and *particle-image* energy (Sect. 3.3) spatially binds particles to creases, while allowing them to move along scale toward maximal feature strength. The constraints and particle-image energy are conceptually similar to the external energy terms of active contours or the data-attachment terms of level sets, which locate image features *Inter-particle* energy (Sect. 3.4) creates uniform feature sampling. Scale-space particles lack an analog to the internal energy of active contours, or the curvature terms of level sets, which ensure smoothness in the feature representation. The particle system does not maintain any particular geometric or topological configuration, as individual particles are independent and unconnected. Rather, constraints keep particles within features, which are smooth curves or surfaces, due to their mathematical definition as well as our scale-space reconstruction (Sect. 3.2).

We minimize the energy of a set of *N* particles {(**x**_{i} *s*_{i})} within the four-dimensional image *f* (**x** *s*), where x is three-dimensional spatial location, and *s* is univariate scale, by numerically solving
TeX Source
$$\mathop {\arg \min }\limits_{\{ (x_i,s_i)\},N} = \mathop {\arg \min }\limits_{\{ (x_i,s_i)\},N} (1 - \alpha)\sum\limits_{i = 1}^N {E_i + {\alpha \over 2}\sum\limits_{i,j = 1}^N {E_{ij} .} } $$
The particle-image energy *E*_{i} (Sect. 3.3) is determined entirely by the local properties of the image *f* at (**x**_{i} *s*_{i}). The symmetric inter-particle energy *E*_{ij} *= E*_{ii} *E*_{ii} = 0 (Sect. 3.4) is independent of location in the image domain. The balance between inter-particle and particle-image energies is determined by the parameter α ∈ [0,1], which is roughly analogous to the user-defined weighting in level-set methods between the smoothness and data-attachment terms.

Not apparent in Eq. 1 are *constraints,* which force particles to remain within features, but which do not themselves contribute to the system energy. Alternatively, without constraints, particles can be steered by an energy that is locally minimal at the feature [63], [29]. For four-dimensional scale-space, the choice between constraints and particle-image energy exists independently for both the three-dimensional spatial domain and the one-dimensional scale domain. Following previous work on implicit surface sampling with particles, we choose to *spatially* constrain particles to lie within crease features. This is enabled by their simple mathematical definition (Sect. 3.3), combined with analytic spatial derivatives of the image (Sect. 3.2). Along the scale axis, however, we use particle-image energy to push particles toward the scales of maximal feature strength. We have found that crease strength measures (Sect. 3.3) are effective but not as stable as the spatial location of the crease itself, so we balance a particle-image energy based on crease strength with an inter-particle energy that attracts particles to each other along scale, producing a feature sampling with better spatial continuity of scale.

### 3.2 Interpolation in Scale-Space

We reconstruct continuous scale-space from discrete samples in space and scale. Each scale sample is some blurring of the original volume data at its original raster resolution. The range of blurring scales is sampled non-uniformly with *S* scales between *s =* 0 (no blurring) to *s =* S_{max}, the largest expected feature scale, which should be known from the imaging application domain, where *s* is the standard deviation of a Gaussian blurring kernel. The scale-space location of a particle and the spatial reconstruction kernels determines the neighborhood of image samples that fall within the spatial convolution support and the two samples along scale that bracket the particle's scale position. We first interpolate, along scale, the spatial support of the particle (Sect. 3.2.1), and then use spatial convolution to reconstruct values and spatial derivatives (Sect. 3.2.2). Following our previous work on anisotropy creases [33], [64], tensors are interpolated componentwise, which facilitates *analytic* computation of spatial derivatives of anisotropy. Non-linear tensor interpolation methods are also possible (*e. g.* [1,3 8]) but lack a similarly accurate way of computing anisotropy derivatives, and with necessarily greater computational expense.

#### 3.2.1 Scale Interpolation

Sampling scale-space by convolving with a blurring kernel whose width equals the scale sample position is completely accurate but prohibitively slow at large scales. Alternatively, one can interpolate between a large set of images pre-blurred at varying scales, and then use a fixed-size spatial support reconstruction in space. For large three-dimensional images, however, the number of such scale samples is limited by available memory. Balancing memory efficiency with reconstruction accuracy is a challenge for volumetric scale-space.

Some previous work has used linear interpolation between preblurred images at adjacent scale samples [34], We introduce a more accurate method using cubic Hermite splines, based on the underlying properties of a particular blurring kernel. Consider first a continuous one-dimensional signal *f*(·) undergoing isotropic diffusion for time *t,* which is equivalent to convolving *f* with a Gaussian *g.*
TeX Source
$$L(x;t)\quad = \quad (f(\cdot)*g(\cdot ;t))(x) = \int {g(\xi ;t)f(x - \xi)d\xi,} $$
TeX Source
$$g(\xi ;t)\quad = \quad \exp (- \xi ^2 /2t)/\sqrt {2\pi t}.$$The time-derivative of *L*(*x*; *t*) is the second spatial derivative [35], [40]
TeX Source
$$\partial L(x;t)/\partial t = {1 \over 2}\partial ^2 L(x;t)/\partial x^2 .$$
For discretely sampled *f* [·], we replace the continuous Gaussian *g* (·; *t*) with its discrete analog *K* [· ; *t*], created by Lindeberg [39].
TeX Source
$$L[i;t]\quad = \quad (f*K[\cdot ;t])[x] = \sum\limits_n {K[n;t]f[i - n],} $$
TeX Source
$$K[n;t]\quad = \quad \exp (- t)I_n (t),$$where *I*_{n} is the modified Bessel function of integer order *n.* By the recursive properties of *I*_{n}, and the design of *K* [·; *t*], the continuous derivative with respect to time is found by second central differences [39]
TeX Source
$$\partial L[i;t]/\partial t = {1 \over 2}(L[\cdot ;t]*[1\quad - 2\quad 1])[i].$$
With a change of variables from diffusion time *t* to spatial scale ,
TeX Source
$$\partial L[i;s^2]/\partial s = s(L[\cdot ;s^2]*[1\quad - 2\quad 1])[i].$$
This analytical derivative ∂ *L*/∂ *s* enables our cubic Hermite spline interpolation. As pointed out by Lindeberg, the same property does not hold for blurring with a discretely sampled continuous Gaussian.

We pre-compute blurrings of the original data *f*[·, ·, ·] with a set of *S* different three-dimensional discrete Gaussians , with non-uniformly spaced scale positions . Blurring at scale *s*ℓ to produce volume *f*[·, ·, ·, ℓ] is computed by
TeX Source
$$f[i,j,k,\ell] = \sum\limits_{m,n,o} {K[m,n,o;s_\ell ^2]f[i - m,j - n,k - o],} $$
TeX Source
$$K[m,n,o;t] = \exp (- 1)^3 I_m (t)I_n (t)I_0 (t).$$To interpolate between *f*_{0} = *f*[*i*,*j*,*k*, ℓ_{0}] and *f*_{1} = *f*[*i*,*j*,*k*, ℓ_{1}] at *s*, we find *s*_{ℓ1} such that *s*_{ℓ1} ≤ *s* < *s*_{ℓ1}, let Δ = (*s* − *s*_{ℓ0})/(*s*_{ℓ1}− *s*_{ℓ0}), and compute the 3-D analog to (8) at *s*_{ℓ0} and *s*_{ℓ1},
TeX Source
$$f'_0 = s_{\ell _0 } (f[\cdot, \cdot, \cdot,\ell _0]*M)[i,j,k],$$
TeX Source
$$f'_1 = s_{\ell _1 } (f[\cdot, \cdot, \cdot,\ell _1]*M)[i,j,k],$$where *M* is the standard 3 × 3 × 3 discrete Laplacian mask. The the cubic Hermite spline between scales is
TeX Source
$$f[i,j,k](s) = [\Delta ^3 \quad \Delta ^2 \quad \Delta \quad 1]\left[{\matrix{ {f'_0 + f'_1 + 2f_0 - 2f_1 } \cr { - 2f'_0 - f'_1 - 3f_0 + 3f_1 } \cr {f'_0 } \cr {f_0 } \cr } } \right]\;.$$
For comparison, linear interpolation would simply be
TeX Source
$$f[i,j,k](s) = (1 - \Delta)f_0 + \Delta f_1 .$$
For each particle, we interpolate along scale only those image samples that fall within the support of the spatial reconstruction kernels.

We measure scale-space reconstruction error as the squared difference between the true blurring kernel (the point-spread function *K*[·, ·, ·, *s*^{2}], and an approximate kernel interpolated between *K*[·, ·, ·, *s*_{ℓ0}^{2}] and *K*[·, ·, ·, *s*_{ℓ1}^{2}], averaged over the support of *K.* Figure 2 depicts reconstruction error from linear and spline interpolation, with uniform and non-uniform scale samples. The locations of the non-uniform scale samples were computed by a brute-force gradient-descent minimization of the mean squared error, numerically integrated over all scales. For a given reconstruction accuracy, our new Hermite spline approach requires significantly fewer pre-blurred volumes *f*[·, ·, ·, ℓ], which greatly reduces memory use and makes continuous scale-space analysis practical for large 3-D data.

#### 3.2.2 Spatial Interpolation and Differentiation

Following scale interpolation, we compute spatial interpolation and differentiation by convolving with separable piecewise polynomial kernels. From the framework of Möller *et al.* [50], we use the six-sample support, fifth-degree polynomial *C*^{3} continuous approximation filter, with fourth-degree error (which exactly reconstructs cubic polynomials). The data is first de-convolved so subsequent first convolution interpolates the original data. We have used a high order of continuity and accuracy so that the creases (themselves determined by second derivatives) are smooth and ripple-free. Continuous values *f*(**x** *s*) are reconstructed by convolving discrete data samples *f* [*i* *j* *k*] (*s*) with the separable filter *R*(*x* *y* *z*) = *r*(*x* *r*(*y* *r* (*z*) for the chosen univariate filter *r*(*x*). Partial derivatives are measured by convolving *f* [*i* *j* *k*] (*s*) with the corresponding partial derivative of *R.*

The final reconstruction step is scale-normalization of derivatives, as described by Lindeberg [41]. Intuitively, at higher blurring levels, the data signal naturally becomes flatter, so spatial derivative magnitudes decrease. To make meaningful comparisons between derivatives at different scales, they are normalized by scale:
TeX Source
$$\tilde \nabla f({\bf{x}},s)\quad = \quad s\nabla f({\bf{x}},s),$$
TeX Source
$${\bf{\tilde H}}f({\bf{x}},s)\quad = \quad s^2 {\bf{H}}f({\bf{x}},s).$$We use the tilde to denote a quantity associated with a scale-normalized derivative. For DTI analysis, scale-normalization is applied to the derivatives of the individual tensor components, not to the derivatives of the derived non-linear tensor attributes like FA.

### 3.3 Particle-Image Energy and Crease Feature Definition

We use Eberly's definition of creases in terms of the gradient **g** =▿ *f* and Hessian **H** of a smooth function *f* [16]. For DTI analysis, the gradient and Hessian of fractional anisotropy (FA) are computed *analytically* from the gradients and Hessians of the individual tensor components, using formulae in [33], Let {
**v**_{i} **v**_{2} **v**_{3}} be unit-length eigenvectors of **H** with corresponding eigenvalues λ_{1} ≥ λ_{2} ≤ λ_{3} Crease surfaces are defined by **g** ·
**v**_{i} = 0 and crease lines by **g** ·
**v**; =
**g** ·
**v**_{j} = 0, with *i* *j* set according to crease type (ridge or valley). For example, ridge surfaces are defined by **g** ·
**v**_{3} = 0 is maximized with respect to motion along the direction of greatest negative curvature. Crease feature strength *h* is quantified by the magnitude of certain eigenvalues of the scale-normalized Hessian *e.g.* is ridge surface strength since a very negative λ_{3} indicates strong downward concavity Table 1 summarizes crease features with the defining dot products, required eigenvalue sign, feature strength *h,* and an *approximate* crease tangent projection T used in particle updates (Sect. 3.5). Particle-image energy *Ei* (1) is defined in proportion to feature strength *h* by
TeX Source
$$E_i = e(f,x_i,s_i) = - \gamma h({\bf{x}}_i,s_i).$$
The negation in (17) makes particles locate higher crease strengths while decreasing their energy. Although total energy (1) has the *α* parameter to balance inter-particle and particle-image energy, we include γ > 0 in (17) to convert from units of the Hessian (length^{–2}) to units of energy, and to control for the dynamic range of image values. Section 3.5 describes a way to set *γ*automatically.

The spatial crease constraints are implemented in terms of definitions in Table 1. If our feature were the implicit surface *f* = 0, the gradient **g** would be used in Newton-Raphson surface localization [70], [45], Creases are local extrema, so we perform a minimization or maximization, rather than a root-finding, to locate them. We restrict gradient ascent (for ridges) or descent (for valleys) to the perpendicular space of the approximate crease tangent **T**.
TeX Source
$${\bf{x}} \leftarrow \left\{ {\matrix{ {{\bf{x}} + c({\bf{I}} - {\bf{T}}){\bf{g}}} & {{\rm{for ridges}}} \cr {{\bf{x}} - c({\bf{I}} - {\bf{T}}){\bf{g}}} & {{\rm{for valleys}}} \cr } } \right.$$
Particles are constrained to creases by iterating (18) with an adaptive stepsize *c* until the length of updates to spatial position **x** falls below ∊_{c} times the voxel size; current work uses ∊_{c} = 0.001. Note that at convergence of (18), (**I** –
**T**)
**g** =
**0** and *I* = Σ_{i}*v*_{i} ⊗*v*_{i} imply the **g** · **v**_{i} = 0 crease definition. The constraint is enforced per particle per iteration, conceptually happening instantaneously relative to the particle motion that minimizes the total system energy (1). (18) does not exactly transport particles onto creases along a perpendicular, but it is accurate enough for the small position updates that arise in our system.

In data with any amount of noise, a crease feature that usefully models some underlying anatomic structure will, at its perimeter, gradually become meaningless as it extends into the background. To keep sampling within well-defined features, we impose a user-specified global minimum crease strength *h*_{min}, which limits the sampled feature extent. Strength thresholding tends to separate the crease into a larger number of disjoint pieces, each with an open boundary. We currently do not explicitly locate the boundary where *h***(x)** = *h*_{min} but simply delete particles that fall below the strength minimum. This has implications for inter-particle spatial repulsion, discussed next.

### 3.4 Inter-Particle Energy

The inter-particle energy determines how particles interact with each other. Previous work has explored varying total potential [48], radius [15], [45], or orientation [62], based on local data properties. Our focus is an initial exploration of particles in scale-space, rather than the optimization of feature sampling, so currently we consider only rotationally symmetric and fixed-size spatial energy profiles. The inter-particle energy at (**x**_{i} *s*_{i}) due to (**x**_{j} *s*_{j}) is
TeX Source
$$E_{ij} = \Phi (r_{ij},s_{ij}) = \Phi \left({{{{\rm{|}}x_i - x_j {\rm{|}}} \over {\sigma _r }}{{s_i - s_j } \over {\sigma _s }}} \right),$$
where σ_{γ} and σ_{s} are user-specified radii along space and scale. These radii are set based on the desired sampling resolutions in space and scale, likely with recognition of the intrinsic resolution of the image itself. We present two energy functions, Φ_{1} for sampling the full scale-extent of a feature, and Φ_{2} for localizing the scale of maximal feature strength. Both have finite support in *r* and *s.*

We start with the single spatial radial energy profile, a tunable C^{2} piecewise cubic function ϕ_{c}(*γ*) with a potential well of depth *d* < 0 at radius *r* = *w,* shown in Fig. 3(a).
TeX Source
$$\phi _c (r) = \left\{ {\matrix{ {1 + {{3(d - 1)|r|} \over w} - {{3(d - 1)|r|^2 } \over {w^2 }} + {{(d - 1)|r|^3 } \over {w^3 }}} \hfill & {0 < |r| < w,} \hfill \cr {d - {{2d(|r| - w)^3 } \over {(w - 1)^3 }} - {{3d(|r| - w)^2 } \over {(w - 1)^2 }}} \hfill & {w \le |r| < 1,} \hfill \cr 0 \hfill & {|r|\; \ge 1.} \hfill \cr } } \right.$$
The negative potential well in ϕ_{c}(*γ*) serves two purposes. First, as creases are often disjoint open surfaces or curves, some attraction maintains a tighter sampling pattern than created by pure repulsion (*i*.*e*. ϕ (*r*) ≥ 0 ∀ *r*), as with the Coulomb [29], cotangent-based [48], and Gaussian [70] profiles used previously. This is the main way in which our crease sampling goal differentiates our inter-particle energy from previous energies for sampling closed surfaces (with the exception of the infinite-support Lennard-Jones potential, also with a potential well [62]). Second, a potential well in ϕ (*r;*) allows population control to be expressed and implemented as energy minimization. As described in Section 3.5, total system energy (1) can be lowered by adding particles at the edge of a well-sampled area or by filling in holes in the distribution. We use *w* = 0.6 and *d* = – 0.002 for this work, as determined by experience, but we have not found the system to be very sensitive to small changes in these values.

Our first energy function Φ_{1}, shown in Fig. 3(b), evenly distributes particles along both space and scale.
TeX Source
$$\Phi _1 (r,s) = \phi _c (\sqrt {r^2 + s^2 }).$$
When using Φ_{1}, we set *α* in (1) to 1.0 (turning off particle-image energy), so that we can visualize the full extent of each feature along scale and see how feature position changes as a function of scale.

However, with Φ_{2}, shown in Fig. 3(d), particles *attract* each other along the scale while repelling in space, with a blending of two terms controlled by *β* ∈ [0,1].
TeX Source
$$\Phi _2 (r,s)\quad = \quad (1 - \beta)\phi _c (r)b(s) + \beta b(r)b(s)s^2,$$
TeX Source
$$b(\omega) = {1 \over {1 + ({\textstyle{\omega \over {b_{{\rm{cut}}} }}})^{2b_{{\rm{ord}}} } }}.$$Together with *α* *<* 1, the particle-image energy (previous section) will draw particles along scale so they coalesce at the scale-space position of peak feature strength while maintaining uniform spatial distribution. Attraction along scale keeps spatially adjacent particles at similar scales, which makes the feature sampling more robust. We use the transfer function of a low-pass Butterworth filter [25] *b*(*ω*) (Fig. 3(c)) to enforce locality along *r* and *s,* not because of any spectral properties, but simply because it is a smooth, tunable approximation to a box function. Current work has used *b*_{ord} = 20 and *b*_{cut} = 0.87.

### 3.5 Initialization, Computation, and Visualization

The goal of initialization is to sample all features, wherever they may be, with at least one particle. Following the general strategy of Jalba *et al.* [29], for *every* voxel, we seed particles at multiple locations along scale (no more than the *S* pre-blurred volumes). From their initial seed location, particles undergo constraint satisfaction (Sect. 3.3) but are deleted if they travel more than two voxel widths before convergence. Particles are then added to the system only if their support does not overlap with more than 50% of a previously added particle's support, and if their feature strength exceeds the user-specified threshold *h*_{min.} The result is a large number of initial particles placed unevenly on whatever creases are present in the scale-space domain. Although an abundance of particles initially slows the particle system, it guarantees the sampling of all features; the alternative of seeding fewer particles risks losing features in exchange for speed, a risk that compromises our ultimate goal of reliable feature extraction.

The mechanics of particle system computation are based on previous work [70], [48], [46], Particles are organized into bins that cover the domain, at least 2σ_{r} along each edge in space, and 2σ_{s} along scale, ensuring that all possible neighbors of a particle are found within its own bin or in a neighboring bin. Each iteration consists of an asynchronous update of each particle and a reapplication of the feature constraint. A particle's update is found by projecting the negative gradient of its energy -▿ onto the approximate local feature tangent **T**, and finding a step along –T▿ that decreases energy . The important details are the energy gradients and approximate feature tangents.

The gradient of the energy of particle (**x**_{i}, s_{i}) is
TeX Source
$$\displaylines{ \nabla _i = \left[{\matrix{ {{{d_i } \over {d{\bf{x}}_i }}} \cr {{{d_i } \over {ds_i }}} \cr } } \right] - (1 - \alpha)\left[{\matrix{ {{{dE_i } \over {d{\bf{x}}_i }}} \cr {{{dE_i } \over {ds_i }}} \cr } } \right] + \alpha \sum\limits_{j = 1}^N {\left[{\matrix{ {{{dE_{ij} } \over {d{\bf{x}}_i }}} \cr {{{dE_{ij} } \over {ds_i }}} \cr } } \right]} \cr = (\alpha - 1)\gamma \left[{\matrix{ {{{\partial h({\bf{x}},s)} \over {\partial {\bf{x}}}}({\bf{x}}_i,s_i)} \cr {{{\partial h({\bf{x}},s)} \over {\partial s}}({\bf{x}}_i,s_i)} \cr } } \right] + \alpha \sum\limits_{j = 1}^N {\left[{\matrix{ {{1 \over {\sigma _r }}{{x_i - x_j } \over {{\rm{|}}{\bf{x}}_i - {\bf{x}}_j {\rm{|}}}}{{\partial \Phi (r,s)} \over {\partial r}}(r_{ij},s_{ij})} \cr {{1 \over {\sigma _s }}{{\partial \Phi (r,s)} \over {\partial s}}(r_{ij},s_{ij})} \cr } } \right]} . \cr} $$The partial derivatives of Φ (either Φ_{1} or Φ_{2} of Section 3.4) are known analytically. The partial derivative of strength *h*(**x** *s*) with respect to *s* is computed numerically, using central differences with step *s*_{max}/1000. For the scale-space crease feature tangent, in the spatial domain we use the approximate crease tangent **T** defined in Table 1, which has worked well even though correct tangents require third derivatives [16]; others have previously used the same approximation [3], Along scale, we assume the feature tangent is (**0**, 1) and thus bypass computing ∂ *h*(*x*,*s*)/∂*x*. This implies that the crease is exactly stable with respect to scale, which is accurate enough when using particle-image energy and Φ_{2} to localize maximal feature strength.

We evaluate the system for population control with a periodicity of PC iterations (currently PC = 10), traversing the particles once for deletion and once for addition. A particle is deleted if the total system energy would be lower without it, or if its strength falls below *h*_{min}
*.* Particles are not deleted if more than half of their neighbors have just been deleted, to avoid overly aggressive culling of dense samplings. New particles are tentatively added next to existing particles that have a markedly asymmetric distribution of neighbors. Each tentative new particle undergoes PC iterations of updates in isolation, and the particle is added if the system energy is lower with it included. The negative potential well in ϕ_{c}(γ) (Sect. 3.4) makes this work. If the new particle moves (during its isolated update) into the potential wells of existing particles, and thus adds to a good sampling distribution, the system energy will be lowered by including it.

When using Φ_{2} to sample maximal strength creases, it is helpful to run the system until approximate convergence with *γ* = 1.0 and *α =* 0.0, turning off all spatial interaction. Once particles are clustered around the scale of maximal strength *γ* can be set with
TeX Source
$${1 \over N}\sum\limits_1^N {{{\partial ^2 E_i } \over {\partial s^2 i}} = {{\partial ^2 \Phi _2 (r,s_{ij})} \over {\partial s_i^2 }}} \Rightarrow - \gamma {1 \over N}\sum\limits_1^N {{{\partial ^2 h({\bf{x}}_i,s)} \over {\partial s^2 }}(s_i) = {1 \over {\sigma _s^2 }}} $$
TeX Source
$$ \Rightarrow \gamma = {{ - 1} \over {\sigma _s^2 {1 \over N}\Sigma _1^N {{\partial ^2 h({\bf{x}}_{i,} s)} \over {\partial s^2 }}(s_i)}}.$$The idea in (25) is to equate the average *concavity* (second derivative) along scale of the particle-image energy *E*_{i} with the concavity of inter-particle energy *E*_{ij} so that the intuitive setting of *α =* 0.5 actually balances the influence of the two energies. The second derivative of crease strength ∂^{2} *h*(*x*_{i},*s*)/∂ *s*^{2} in (26) is computed numerically.

While multiple parameters control the particle system, only a few need to be tuned depending on sampling goals. If using scale-attractive inter-particle energy Φ_{2} to localize sampling along scale, the main parameters are: *α* to balance particle-image and inter-particle energies (Sect. 3.1 *β* to balance spatial repulsion and scale attraction (Sect. 3.4), and 7 to modulate energy from feature strength (Sect. 3.3), though *γ* can be set automatically by (26) above. When using scale-repulsive inter-particle energy Φ_{1} to sample the scale extent of features, we set α = 1 (no particle-image energy), and *β* and *γ* are moot. The minimum feature strength *h*_{min} (Sect. 3.3) prevents particles from sampling insignificant features in noise. Interactively thresholding glyph visualizations of particle solutions on small test volumes can inform setting *h*_{min}, though future work may automate setting *h*_{min}. Finally, the particle radii in space σ_{γ} and scale σ_{s} are set based on the desired sampling density relative to the expected characteristic size of features. The other system parameters have not been varied for our different applications, but are listed here for completeness: constraint satisfaction convergence threshold ∊_{c}=0.001 (Sect. 3.3), well position *w* = 0.6 and depth *d* = –0.002 of univariate potential ϕ_{c}(γ), Butterworth order b_{ord}=20 and cut-off b_{cut}=0.87 (Sect. 3.4), and population control periodicity *PC* =10 (above).

Glyph-based visualizations are important for inspecting scale-space particles. Visualizations can be used to debug the system implementation, observe the computation in progress to assess the appropriateness of the parameter settings, and, as we do in the following figures, to evaluate the spatial configuration of the solution and its relationship to cutting planes. While later work will compute geometric models from the particle solutions for further quantitative study, at this stage, the visualizations offer evidence that the relevant anatomical structures are indeed sampled by the particles. Glyphs can be displayed at particle spatial locations {
**x**_{i}} or, following previous illustrations of scale-space [68], [41], the scale axis can be mapped to a user-specified unit-length vector s, with the glyph for (**x**_{i} s_{i}) placed at , with some user-specified scaling *ρ*. Points or other sprites would suffice for depicting the over-all system configuration, but we propose two types of glyphs **D**_{1} and **D**_{2}, which usefully reflect aspects of the eigensystem of the Hessian.

The first glyph **D**_{1} depicts Hessian eigenvectors and the relative magnitude of the eigenvalues. Existing tensor glyphs only express positive eigenvalues, so from the eigenvalues λ_{i} and corresponding eigenvectors **v**_{i} we find new positive eigenvalues μ_{i} for a tensor **D**_{1}, which we visualize with superquadric glyphs [32].
TeX Source
$$\lambda ' = (|\lambda _1 | + |\lambda _2 | + |\lambda _3 |)/10,\quad \lambda ''_i = 1/\max (\lambda ',|\lambda _i |),$$
TeX Source
$$\mu _i = \lambda ''_i /\max (\lambda ''_1,\lambda ''_2,\lambda ''_2),\quad {\bf{D}}_1 = \sum\limits_i {\mu _i {\bf{v}}_i \otimes {\bf{v}}_i .} $$By using the reciprocal of the Hessian eigenvalues, the smallest magnitude eigenvalues λ_{i} associated with the eigenvectors that approximate tangents to the crease feature, become the largest μ_{i}, so that the glyph visualizes the approximate local feature orientation. The glyph shapes along crease line are thus roughly cylindrical (aligned with the crease), and crease surfaces are indicated by disc-like glyphs, while the over-all size is controlled so the maximum μ_{i}, eigenvalue is one.

The second glyph **D**_{2} explicitly indicates a particle's scale position s_{i} with glyph size along the eigenvectors *not* associated with the approximate crease tangent **T** (Sect. 3.3)
TeX Source
$${\bf{D}}_2 = {\bf{T}} + s_i ({\bf{I}} - {\bf{T}})$$
The base geometry is a cylinder with the axis of rotation aligned along either the principal or the minor eigenvector of **D**_{2}, based on whether **D**_{2} is more linearly or more planarly anisotropic, according to measures *c*_{l} and *c*_{p} [67], as in Fig. 3 of [32], Narrow cylinders, approximately tangent to crease lines at small scales, fatten into discs at larger scales; conversely, thin discs for crease surfaces extend into longer cylinders at larger scales.

The final component of our particle system visualization is the extraction of "connected" components. Even though there is as yet no explicit connectivity between particles, they can still be organized into equivalence classes by (*x*_{i},*s*_{i})∼(*x*_{j}, *s*_{j}) iff *E*_{ij} ≠ 0. In practice, the inter-particle energy *E*_{ij} is non-zero for any two particles with overlapping support, even though the potential functions have zero-crossings. Particles are thus "connected" if they are near enough in space and scale to interact; disjoint features tend to be sampled by particles in different connected components. This is an imperfect means of isolating features because particles from different features can occasionally interact. We have found the connectivity heuristic, combined with sorting the components by number of particles, to help inspect the main image features sampled by the particle computation.

Figure 1 demonstrates scale-space particles on a synthetic variable-width torus (Fig. l(a)), in which single-scale ridge sampling is clearly insufficient (Fig. l(b), second row Fig. l(c) extends scale vertically, showing how the Φ_{1} inter-particle energy (with zero particle-image energy) samples the complete feature manifold in scale-space, visualized with **D**_{1} glyphs color-mapped with crease strength Fig. l(d) uses *α* = 1 (no inter-particle energy) to condense all the particles to the scale of maximal strength. Applying population control with Φ_{2} and *α = β = 0.5* Fig. l(f) shows the final result with **D**_{2} glyphs. Using linear instead of spline interpolation between scales, with the same number of pre-computed blurrings, significantly impedes accurate feature scale localization (Fig. l(e)).

Fig. 4 illustrates non-scale-space properties of our particle system with a Möbius strip that emphasizes how crease surfaces are in general non-orientable [58], The right side of Fig. 4 includes a portion of a cutting plane showing the coarse resolution of the underlying scalar data. The **D**_{1} glyphs show both the over-all shape of the Mobius strip and the shape of the Hessian at each particle. Through the middle of the Möbius strip, the **D**_{1} glyphs (28) indicate that the approximate tangent **T** (Table 1) seems to accurately reflect the actual surface tangent, which helps justify the constrained gradient ascent or descent approach of constraint satisfaction in (18). Away from the middle of the strip, the glyphs diverge most from a circular disc shape, and the ridge surface strength (indicated by the colormap) decreases. The errant, semi-transparent glyphs in Fig. 4 fell below a user-specified feature strength threshold. This illustrates how creases can be sampled even where poorly defined, and it emphasizes the need for strength thresholding to limit particles to meaningful features.

Valley lines indicating lung airways were sampled in the left lung of a 0.5mm^{3} resolution chest CT scan (a 230 × 290 × 480 subvol-ume), with eight samples along scale *s* ∈ [0mm, 8mm], using Φ_{2} with *α* = 0.5 *β* = 0.5, σ_{r} = 1.2mm, σ_{s} = 2.5mm Figure 5(a) uses **D**_{2} glyphs color-mapped by strength to show the result of 6.4 hours of processing on 2.8 GHz Intel Core 2 Duo (single-threaded). With strength thresholding and connected component analysis, the wide range of scales in the branching structure is shown in Fig. 5(b), which includes for comparison a region-growing segmentation, shown as a semi-transparent surface. Note that the glyphs and the segmentation surface agree in their indication of both the airway location and radius over the full range of scales. Small airways are difficult to capture by region-growing methods due to limited resolution, even though they are especially valuable for quantitative study [28] Figure 5(b) shows how particles tend to capture *greater* extents of the fine-scale airways, compared to the segmentation, while they also naturally capture even the widest airways. The cutting plane in 5(c) illustrates the underlying unsegmented data quality and resolution relative to the features successfully sampled within it.

The scale-repulsive inter-particle energy Φ_{1} permits visualizing the complete scale-space extent of a feature Figure 6(a) shows this for a single airway vessel segment, while colormapping onto round glyphs the valley line strength. This type of visualization informed our choice of inter-particle energy rather than constraint satisfaction for feature localization along scale, because the visualization showed the high spatial variability of feature strength Figure 6(b) uses the scale-attractive inter-particle energy Φ_{2} with low values of *α* and *β* (so that particle-image energy dominates inter-particle energy, which is mostly spatially repelling), to produce what is essentially a plot of the highly variable scales of maximal strength along the feature. Higher values of *α* and *β* in Fig. 6(d) (*i.e.* scale-attractive inter-particle energy dominates the particle-image energy) regularize the spatial variability of scale. There are noticeable but not dramatic improvements in the resulting sampled feature uniformity, shown in Fig. 6(c).

Figure 7 shows single-subject FA ridge surface analysis in a [128 × 128 × 105] 1.5mm^{3} resolution DTI scan, sampled six times along *s* ∈ [0mm, 5mm], Colormapping by scale illustrates how features near the cortical surface are better sampled at fine scales, while larger scales are better for structures deep inside the brain. The computation required roughly 8 hours, using Φ_{2} with *α* = 0.7 *β* = 0.5 *σ*_{r} = 2mm *σ*_{s} = 2.2mm Figure 8 demonstrates why scale-space analysis is useful for brain DTI: the corpus callosum (the thick connection between the hemispheres) is not reliably represented by an FA ridge surface sampled at a single scale, appropriate for structures closer to the cortical surface, while its interior shape is well sampled even at the thickest part by the scale-space particles.

Complementing FA ridge surfaces are the FA ridge lines in Fig. 9, shown with D_{2} glyphs that indicate the sampled lines as variable-width tubes. While we are still understanding which of these lines correspond to major white matter pathways Fig. 9(b) shows that certain major tubular pathways, such as the Fornix and Cingulum Bundles, seem to be well captured. Given the generic nature of scale-space particles, we feel this represents a useful contribution to brain DTI analysis, given that other methods have been designed specifically to create models of just these pathways [51].

SECTION 5

## Conclusions and Discussion

Scale-space particles combine approaches from graphics, visualization, and vision to create a tool for robustly sampling and visualizing structure in real-world volume datasets. From computer vision, we leverage Lindeberg's discrete Gaussian [41] to create a memory-efficient scale-space reconstruction, and use scale-normalized strength measures [41] for crease feature [16] localization along scale [69], [35], [66], The basic mechanics of particle-based feature sampling were derived from computer graphics [62], [70], [27] and scientific visualization [48], [46], and the whole system was debugged, monitored, and evaluated here with tensor glyph visualizations [32], Our results suggest that many possible biomedical applications of particle systems have yet to be explored, and that scientific visualization provides an effective framework for those explorations. Without being particularly adapted for either lung or brain analysis, our particles capture airways from CT as well as white matter structure from DTI, due to the particles' ability to gracefully handle features that span a continuous range of scales. The increasing resolution of medical scanners will decrease the lower limit on feature scale, creating more demand for better feature sampling and visualization methods that, like scale-space particles, work naturally over a wide range of scales.

There are many possible directions for future research. The refinement of particle systems for analyzing the scale-space structure of non-scalar data is especially exciting. It may be possible to reliably generate useful models of major white matter pathways solely from the differential structure of a single diffusion tensor invariant. This is counter-intuitive, given that we are not analyzing what might seem to be the most important aspect of the tensor, the principal eigenvector that is the basis of fiber tractography. On the other hand, we are implicitly taking advantage of significantly more tensor information by blurring the tensor field and then measuring anisotropy (rather than vice versa). Exploiting the non-linearity of anisotropy in this way mirrors earlier work on measuring tensor field *coherence* by blurring and then measuring anisotropy [67], Future work can explore the non-linearity of anisotropy with respect to scale in more detail. The system described here may in fact be the most practical tool to help explore the broader question of the precise anatomic relationship between FA creases extracted in scale-space, and the shape of major pathways as computed from tractography.

Another direction for future work is the reduction of the number of particles by tuning the sampling density to local feature properties. Previous work [45] did this with a medial axis of the closed surface, but we see no obvious analog to medial axes for creases: crease lines are codimension-two, and crease surfaces can be non-orientable [58], It is tempting to make the spatial particle radius vary in proportion to the scale position (larger particles at higher blurring levels), but it is unclear whether scale actually puts an upper bound on feature curvature, especially for creases of non-linear anisotropy measures.

Finally, there are many ways of optimizing our proof-of-concept implementation. Though we now err on the side of seeding too many particles, a secondary strength threshold (lower than the *h*_{min} in Sect. 3.3), evaluated at starting seed locations, could accelerate initialization and reduce the initial population. The current bottleneck in our system is actually in the later stage of computing particle motion, after population control has handled feature oversampling. Constraining particles to creases is expensive due to the many convolutions needed to reconstruct values and derivatives (Sect. 3.2). Simply copying the image values within the kernel support from the volume data into a convolution buffer accounts for a large part of the cost; it would be alleviated by more efficient use of the memory hierarchy through volume bricking [53], Given the data-parallel nature of convolution and constraint satisfaction, multi-threading would also be beneficial, as would a GPU-based approach. Correctly parallelizing inter-particle interactions in the context of the global data structure used for spatial binning will require some care.