IEEE Xplore At-A-Glance
  • Abstract

Volume Illustration of Muscle from Diffusion Tensor Images

Medical illustration has demonstrated its effectiveness to depict salient anatomical features while hiding the irrelevant details. Current solutions are ineffective for visualizing fibrous structures such as muscle, because typical datasets (CT or MRI) do not contain directional details. In this paper, we introduce a new muscle illustration approach that leverages diffusion tensor imaging (DTI) data and example-based texture synthesis techniques. Beginning with a volumetric diffusion tensor image, we reformulate it into a scalar field and an auxiliary guidance vector field to represent the structure and orientation of a muscle bundle. A muscle mask derived from the input diffusion tensor image is used to classify the muscle structure. The guidance vector field is further refined to remove noise and clarify structure. To simulate the internal appearance of the muscle, we propose a new two-dimensional example-based solid texture synthesis algorithm that builds a solid texture constrained by the guidance vector field. Illustrating the constructed scalar field and solid texture efficiently highlights the global appearance of the muscle as well as the local shape and structure of the muscle fibers in an illustrative fashion. We have applied the proposed approach to five example datasets (four pig hearts and a pig leg), demonstrating plausible illustration and expressiveness.

SECTION 1

Introduction

Muscle, an important organ of the human body, produces force and causes motion. Illustrating muscles [34] is useful in many applications such as medical education, cardiology, functional human movement prediction, and planning surgery for patients with musculoskeletal disorders. The muscles in the human body can be classified into three categories [5]: (1) skeletal muscles that are attached to bone by ten-dons and influence skeletal movement, (2) cardiac muscle that makes up the heart, and (3) smooth muscle that exists at various scales in almost every organ. The structure of these three types of muscles is quite different. Skeletal and cardiac muscles consist of highly regular bundles, yet smooth muscles do not have obvious fiber structures. The isotropic patterns in smooth muscles can be simulated by standard solid texture synthesis techniques [24], [36]; whereas structural muscles including skeletal and cardiac muscles require special consideration because they are composed of a set of oriented fiber bundles. The graphics and bio-mechanics communities have widely studied the simulation of skeletal muscle by using geometric techniques [30], [37] and anatomical information [5]. Their results are normally displayed as geometric models and are not amenable for depicting internal appearance.

Another class of methods [13], [14] detects oriented edges from CT or photographs and constructs a solid texture, opening new opportunities for volumetric representation and simulation of muscles. Unfortunately, the orientation information computed by edge detection and Hough transformation may not be physically accurate, and the employed texture synthesis technique tends to produce non-optimized results, yielding vague fibrous structures.

Diffusion tensor imaging (DTI) has been shown to be able to reconstruct 3D fibrous structures such as muscles [31], [32] and neural fibers in the brain [12]. In this paper, we visualize muscles by leveraging the DTI and example-based texture synthesis techniques with two main contributions:

  1. We reformulate the fiber bundles constructed from DTI data into a volumetric representation and employ a new example-based technique to synthesize a solid texture to encode the structure and appearance of muscles.

  2. We employ a line-based volume illumination equation modulated with a synthesized volume and a solid texture to simultaneously depict the structure and appearance of muscles.

Our approach is conceptually different from appearance-oriented volume synthesis approaches [24], [25], [26] because the generation of the solid texture is guided by the principal vector field built from a DTI data set. Our approach also takes a different form than the LIC-based DTI visualization approach [20] in that ours facilitates characterizing individualized muscle structure by means of a scalar field while providing contextual muscle appearance with the synthesized solid texture.

We consider setting up a framework for achieving realistic muscle illustration as a major contribution of this paper. With our system, we demonstrate how to construct illustrative depiction of fibrous structures from 2D examples and DTI images. Our approach provides an expressive visualization of muscle and is an effective complement to previous geometry- and biomechanics-based visualization methods. A cardiologist confirms that our method shows both the normal heart muscle fibers and the infarcted areas well.

The rest of this paper is organized as follows. We discuss related work in Section 2 and outline the algorithm in Section 3. Computing two fields based on a DTI image is explained in Section 4. Section 5 describes the constrained solid texture synthesis and the illustrative volume illustration of the constructed muscle volume. In Section 6, we present the experimental results and discussions. Finally, Section 7 concludes the paper.

SECTION 2

Related Work

Muscle Visualization In the computer graphics community, the focus of muscle visualization is on the boundary shapes of muscles and their animation. For instance, Scheepers et al. [30] considered the influence of the musculature on surface and generated muscle models that react automatically to changes in the posture of an underlying articulated skeleton. To achieve high accuracy of muscle and tendon dynamic deformation, Teran and his colleague [37] presented a framework for extracting and simulating high-resolution musculoskeletal geometry from the segmented Visible Human data set.

The internal structure and appearance of muscle is of great interest for medical education, cardiology, and functional human movement prediction. A common method is volumetric texture synthesis that is capable of simulating the repeated patterns in solid objects [25], [26]. To effectively depict the oriented fiber structures, Dong and his colleague [13], [14] detected oriented edges from CT or photographs datasets and employed a patch-based solid texture synthesis algorithm. Hsu [20] utilized the local tensor information from a DTI dataset, and employed a generalized line integral curve algorithm to visualize both the principal and second-order anisotropy of the cardiac muscle. Although the texture can be displayed with volume rendering techniques, it is not effective in illustrative rendering of individual fibers because the structure information encoded in the LIC texture is not sufficiently strong. In this paper, we encode the DTI fibers in a scalar field to emphasize the fibrous structures, and we apply a new solid texture synthesis technique to depict the internal appearance.

Alternatively, one can utilize the glyph- or streamline-based rendering techniques to directly show the fiber structures [18], [27], [45]. Selected coloring schemes can be used to show the local tensor or the spiral orientation information [45]. The results from the stream-lines/streamtubes methods [29], [45] on heart muscle DTI data show the orientation of the myocardial muscle fibers including the helical patterns in the heart muscle. However, the concerns about the interactive speed and the visual cluttering restrict the density of the streamtubes, and the resulting streamtubes models do not emulate the appearance of the muscle fibers. In this paper, we aim to emulate the appearance of the muscle fibers as well as to represent their trajectories.

Volume Illustration Over the past decade, volume illustration has attracted much attention [6], [15], [25] for its effectiveness to depict salient anatomical features while hiding the unimportant details. Standard approaches take a volume dataset as input and exploit multi-dimensional transfer function design to highlight important features and subjugate insignificant details. More comprehensive effects can be achieved by illustrating internal appearance [26] and volumetric manipulation [10] and learning color, texture, or lighting styles from examples [25]. However, there is not an approach that is capable of visualizing the fibrous structures of muscles in an illustrative fashion. Challenges for this goal include how to construct the shape and structure of muscles and how to illustrate the internal appearance with quality comparable to hand-drawn illustrations.

Diffusion Tensor Imaging Visualization Fibrous structures such as the muscle or the brain white matter constrain the water diffusion in the directions orthogonal to the orientation of the fibers, or constrain the diffusion such that it is mainly along the fiber directions. Therefore, the diffusion information collected by DTI can be used to reconstruct the trajectories of muscles and the neural fibers [3]. The reconstructed fiber orientations have been validated in both cardiac muscle [31] and skeletal muscle [32] and can be used for assessments of tissue health like cardiac analysis [28]. Filters have been applied to remove noise in diffusion-weighted imaging data [1], [4] or regularize diffusion tensor imaging data [11], [38]. In this paper, we employ a simple filter to enhance the illustration of the muscle fibers.

Various approaches have been applied to visualize 3D DTI data [40] including glyphs [23], superquadric glyphs [18], volume rendering [22], streamlines/streamtubes [17], [27], [29], [39], [44], and the combination of volume rendering and streamtubes methods [8], [42].

Volumetric Texture Synthesis Texture synthesis denotes the process of algorithmically constructing a large texture from a small sample texture by taking advantage of its structural content. As texture can be arranged within a spectrum from stochastic to regular, most work on 2D texture synthesis can be classified into the non-parametric [16], [41] and parametric [19] approaches. These 2D methods can be directly extended to volumetric texture synthesis by taking 3D texture samples as the input. Representative work includes [25], [41]. Because 3D texture samples are not easy to capture, it is preferable to employ 2D examples as demonstrated in [9], [21], [24]. To depict the essential shape character, several researchers [14] sought to perform the synthesis guided by a directional field. Dong and his colleague [14] synthesized a volumetric texture based on the extracted orientation field from the Visible Human data. A more recent work [36] presents an interactive modeling system for building spatially varying oriented solid textures by repeatedly pasting 3D texture exemplars within a tetrahedral mesh. We use diffusion tensor images to produce anatomically relevant results that allow visualization of muscle from multiple perspectives.

SECTION 3

The Key Idea

The entire volume illustration approach consists of two stages. In the first stage, we generate a binary volumetric mask that indicates the muscle regions and compute the principal vector field from the input diffusion tensor image. We also build a set of fiber tracts using a standard fiber tracking algorithm [40]. Then we convert the fibers into a scalar field to represent the structure of fiber tracts and construct a refined guidance vector field.

In the second stage, we synthesize a solid texture based on the guidance vector field and an example texture. The scalar field and solid texture constitute the final volume for illustrative rendering Figure 1 depicts the entire pipeline.

Figure 1
Fig. 1. The pipeline of muscle volume illustration from DTI images
SECTION 4

Computing the Scalar and Guidance Fields

In this section we describe how to generate a scalar field and a refined guidance field to encode the needed muscle information.

4.1 The Mask Volume

The conventional DTI fiber model gives a geometric abstraction of the input DTI and forms a continuous representation of the muscle. However, some fibers may result from non-muscular tissues or noise.

To specify the muscular regions, we build a binary mask volume based on the input DTI in three steps. First, we apply the Brain Extraction Tool [33] to the T2 non-diffusion-weighted image to exclude noise from the air surrounding the subject. Second, we set a threshold tm on the T2 non-diffusion-weighted image to exclude fluid in the data volume. Third, we exclude voxels whose linear anisotropy (defined in [43]) is below 0.1. The remaining voxels after these steps are assigned 1 in the mask volume. Other voxels are assigned 0.

DTI fibers are generated by integrating curves in the principal eigenvector field of the diffusion tensor field [44]. The integration is restricted to the regions where the masks are 1. The culling distance between fibers (Dt in [44]) was set to 1.0 mm. The minimum distance threshold (Tt in [40]) was set to 0.5 mm.

4.2 The Scalar Field

Each generated fiber is represented with a set of line segments and is sequentially voxelized within a volume dataset. We first convert each line segment into a sequence of points with a 3D digital differential algorithm (DDA). Then, each point p is splattered into the volume space. The voxels whose distances to p are less than a given threshold ts (typical choice for ts is 3 times the dimension of a voxel) form an influence field of p. We loop over all voxels v in the influence field and compute the distance dv from v to the underlying line segment. A scalar value is computed for each voxel: Formula TeX Source $$s_v = {1 \over {\tau ^2 }}e^{ - k_s^2 d_v^2 /\tau ^2 } $$ Here τ denotes the standard deviation of the exponential kernel. A typical choice is 1.0 ks is a parameter to modulate the attenuation of the exponential kernel. It could be larger or smaller, depending on the distribution density of fibers and the volume resolution. For a dense fiber distribution and a volume resolution of 5123 ks = 2 .0.

The collection of sv of all voxels encodes the fibrous structure of the muscle. It is denoted as the scalar field (see Figure 1), where each voxel records a scalar value.

For efficient abstraction, we neglect fibers whose total lengths are less than a unit voxel length at the given volume resolution. We also perform simplification to each individual fiber by joining adjacent line segments whose orientations are very similar: two line segments are joined if the cosine of the angle between them is larger than 0.98.

4.2.1 Overcoming Ambiguity

If two fibers are spatially close, it is very likely that a voxel belongs to the influence fields of both fibers. We overcome this ambiguity by sequentially determining conflicting fibers and removing them in each fiber bundle. For each fiber, we check its distance to other fibers. If the largest one is less than 0 .85 ts, we remove this fiber and continue the removal operation for the remaining fibers until all fibers are checked. On average, about 5% of the fibers are removed using this criterion Figure 2 (b) illustrates a case in which the yellow one is removed from Figure 2 (a).

The fiber distance based scheme cannot guarantee that all conflicts are eliminated. In those cases, we preserve both fibers and simply assign the values with larger sv (see Equation 1) to the underlying voxels. An example of this type of case is the pair of the red and green fibers shown in Figure 2 (b).

Figure 2
Fig. 2. Illustrating the fiber conversion in 2D. Each filled sphere denotes a fiber tract. (a) The initial result; (b) The yellow one is removed to overcome ambiguity.

4.3 The Guidance Vector Field

We extract the eigenvector associated with the maximum eigenvalue of the tensor in each voxel. The set of eigenvectors constructs a vector field, called the guidance vector field. Artifacts may appear in the synthesized solid texture where the guidance vectors are not defined due to noise or error in the data. Because our goal is to enhance the appearance of the illustration, we choose to use a rather simple filtering method over the guidance vector field instead of regularizing the diffusion tensor field.

4.3.1 Refining the Guidance Vector Field

Computing a smooth and continuous vector field consists of three steps: outlier removal, resampling, and smoothing. In the first step, we seek to remove the voxels whose guidance vectors are significantly inconsistent with their neighborhoods. We loop over each individual voxel v whose mask is 1 and build its 9 × 9 × 9 neighborhood N(v) = vi(i = 1 , 2 ,...729). For the guidance vectors Formula with respect to the voxels Formula in N(v) whose masks are 1, we compute a vector Formula by finding a voxel vr: Formula TeX Source $${\rm{r = argmini \{ }}d_i {\rm{, }}d_i {\rm{ = median }}(||\vec l_i - \vec l_j ||,\,\,j = 1,\;2,...k,j \ne i)\} $$ where median(·) denotes the median value among a set of values.

A voxel vi is defined as an outlier voxel if Formula where K is an adjustable threshold and is set to be rather large (e.g., 20.0) to allow for the inclusion of anatomically reasonable changes in the guidance vector field in our experiments.

Removing the outlier voxels may cause small holes in the muscle. Meanwhile, the regions where the masks are zero normally contain disordered vectors. Thus, to enable smooth boundaries around the muscle regions in the synthesized solid texture, we additionally perform a resampling process. For each voxel v whose mask is zero or each outlier voxel v whose mask is non-zero, we determine Nr (set to be 10) nearest voxels that are not outlier voxels and whose masks are non-zero. We compute a set of weighting coefficients: Formula TeX Source $$\alpha _i = exp(- {{d_i^2 } \over {\bar d^2 }}),\;\beta _i = {{\alpha _i } \over {\sum\nolimits_{i = 1}^{N_r } {\alpha _i } }},0 < \alpha _i,\;\beta _i < 1,\;\sum\limits_{i = 1}^{N_r } {\beta _i = 1,\;i = 1,2,...N_r } $$ where di is the distance from the ith voxel to v,and d-2i denotes the average of the set of d2i

Then, we compute the guidance vector Formula of v by Formula.

Finally, we perform a global Gaussian smoothing operation to make the vector field smoother and more continuous. The width gw of the Gaussian smoothing is set to be 5 Figure 3 demonstrates the effects of these refinements by visualizing the vector fields with the line integral convolution (LIC) [7] technique. Note that, only the regions whose masks are 1 in the refined guidance vector field are used in the volume illustration stage.

SECTION 5

Volume Illustration of Muscles

5.1 Constrained Solid Texture Synthesis

In [24], a non-parametric algorithm for synthesizing a solid texture from 2D examples is proposed. It is computationally efficient and well suited for synthesizing multichannel textures, with or without correlation between the color channels. Here we show how to make it amenable for synthesizing muscle textures constrained by a given vector field.

5.1.1 Iterative Solid Texture Synthesis

For the sake of clarity, we first briefly summarize the algorithm presented in [24]. The synthesis begins with a volume whose colors are randomly chosen from the 2D example. A two-phase optimization is iteratively carried out to minimize the sum of the difference under r norm (r is set to be 0.8) between each local region syi(i = x, y, z) of a voxel sv and corresponding region evi in the 2D example E: Formula TeX Source $$E(s,\;e) = \Sigma _v \Sigma _{i \in x,y,z} ||s_{v,i} - e_{v,i} ||^r .$$

In the first phase, the matching neighborhood of each voxel is fixed, and each voxel is optimized. The second phase searches the best matching exemplar neighborhoods for each voxel. For more details, please refer to [24].

Figure 3
Fig. 3. Demonstration of the vector field refinement with a phantom model. (a) The input is a 3D vector field at the resolution of 2563;(b) A randomly distorted vector field based on (a); (c) The result after refinement. Three vector fields are visualized with the LIC technique. (d) A slice from the input guidance vector field of a normal pig heart. The yellow sticks denote the ones whose masks are 0. The corresponding slice in the synthesized solid texture is shown in the right part. (e) The corresponding results after the refinement to the vector field.
Figure 4
Fig. 4. Matching the neighboring region (along z axis) in the 2D example with standard solid texture synthesis (a) and constrained solid texture synthesis approaches (b)

5.1.2 Vector Field Guided Solid Texture Synthesis

The key idea of vector field guided synthesis is straightforward: in the searching phase, the employed local region evi(i = x,y,z) is rotated around its center by an angle. The angle is decided by the associated guidance vector in the given location. In practice, we pre-compute a sequence of rotated images from the input 2D example and perform the search among these rotated versions. Typically 60 pre-computed directions are adequate. Given a guidance vector Formula in voxel v, we choose a local region evi^* from three 2D examples Ei, which are rotated from E by three angles evi,(i = x,y,z) : Formula TeX Source $$\theta _{v_x } = \arctan \left({{{\vec g_{v,\;y} } \over {\vec g_{v,\;z} }}} \right),\,\;\theta _{v_y } = \arctan \left({{{\vec g_{y,x} } \over {\vec g_{v,z} }}} \right),\;\theta _{v_z } = \arctan \left({{{\vec g_{v,x} } \over {\vec g_{v,\,y} }}} \right)$$

To ensure the rotated example has a rectangular shape, we enlarge the size of each rotated example by √2, as demonstrated in the top left of Figure 4 (b). When Formula, is a zero vector (θvi = 0(i = x,y,z)), we use the input two-dimensional example.

This scheme is simple and efficient if two requirements are met. First, the variations of the guidance vectors in a local region are adequately small. When the guidance vectors are non-smooth or even discontinuous, searching the neighborhood in the rotated 2D examples will lead to conflicting results. We address this problem by performing a smoothing operation to the guidance-vector field, as described in Section 4.3.1. Second, the input 2D example implicitly contains a unique global orientation so that one of its rotations approximately matches a guidance vector. Our system allows the user to specify a 2D example and compute a principal axis for each input 2D example, which is then rotated to align its principal axis with the x (or y)axis.

Figure 5
Fig. 5. (a) An example texture and the result from a solid texture synthesis [24]. Even though the example is applied in three directions, the internal appearance exhibits some distortions. (b) An example texture and the result with our approach. This result clearly depicts a concentric spherical solid pattern that matches the input guidance vector field. (c) An example texture with red and white stripes; a guidance vector field produced from the input principal vector field (each vector is visualized with a color-mapped stick glyph); the synthesized solid texture under a cutting plane view. The solid texture pattern closely follows the integral curves of the DTI dataset.

Figure 5 (a-b) compares the effects by the algorithm in [24] and our approach with a phantom model. For the former, a concentric circle pattern image is used as the input example. Even though the example is applied in three directions, the result is unsatisfying. Our approach employs a red-white stripe image, and an additional guidance vector field, whose vectors point inside to a spherical center. The synthesized result clearly depicts a concentric spherical solid pattern. Based on the same red and white stripe image and a guidance vector field induced from a real DTI dataset, a DTI-driven solid texture is generated as described in Figure 5(c).

5.2 Illustrative rendering of muscle volume

We pack the synthesized volume texture and the scalar field into a solid volume. The volume illustration of the muscles consists of two parts, namely, the internal appearance from a solid texture Cs and a gradient-modulated lighting Cl based on the fiber structure embedded in the scalar field. For Cl at each sampling point, we adopt the line-based illumination model in [2]: Formula TeX Source $$C_l = k_a + C_t (k_d (\root m \of {1 - (\vec v \cdot \vec L)^2 }) + k_s (\sqrt {1 - (\vec v \cdot \vec H})^2)),\;\alpha _l = \alpha _t $$where ka kd, and ks are the coefficients for ambient, diffuse, and specular components. (Ctt) is the color and opacity determined from the user-adjustable 2D transfer function that is dependent on the density and gradient magnitude of the computed scalar field (see Section 3.2). A typical choice for the 2D transfer function is a Gaussian function embedded in a rectangle that covers the valid region of the underlying scalar field in its 2D histogram. The three vectors Formula, and Formula denote the fiber tract direction stored in the shape volume, the light direction, and the halfway vector between the lighting and viewing directions. The two constants m and n express the excess-brightness diffuse and shininess exponents respectively.

The weighted composition of Cs and Cl yields the illumination C = wCs + (1 − w Cl and opacity α = wαl + (1 − w). The contributions of all sampling points are composed in the order of back to front by means of the 3D texture slicing volume rendering algorithm Figure 6 (b-d) demonstrates the volume illustrations for a DTI dataset of a normal pig heart. A red and white strip picture and the computed guidance-vector field of the DTI data are used. From (b) to (d), the weight w is 0.0, 0.5, and 0.75 respectively. By adjusting the composition of the solid texture and the volume illumination, it is convenient to depict the line structure and appearance pattern (see Figure 6 (e-f)).

Figure 6
Fig. 6. Visualization of a normal pig heart. (a) Illuminated line rendering; (b-d) Cutting plane views of the volume illustration with different weights: 0.0, 0.5, and 0.75. (e-f) Another view to the volume illustration with different weights: 0.5, and 0.0. The composition of the solid texture in (e) makes the fiber structure easier to be recognized, and shows the orientation of the major eigenvectors in the part of the DTI field where the fibers do not exist due to the thresholding or culling process, as demonstrated in the regions with yellow circles.
SECTION 6

Results and Discussions

We have tested our algorithm on a sequence of datasets, including four pig hearts and a pig leg. Among the four heart examples, two hearts are from normal pigs, and the other two suffer from induced myocardial infarctions.

Our co-authors include one cardiologist, three doctors of veterinary medicine, and a bio-mechanics researcher. They have tested the presented approach and studied the results. They were generally satisfied with the performance and the interactivity of our visualization system. We include some of their specific comments in the following sections.

6.1 Performance and Parameter Controlling

The implementation was built upon the IVIS volume illustration system [35], whose rendering kernel is a GPU-based 3D texture slicing volume rendering algorithm with the Nvidia Cg language. The slicing number is set to be 800 to obtain high rendering quality. We utilize two-dimensional transfer functions to modulate the line-based volume illumination and compose the results with those from the solid texture. All results in this paper and the companion video are rendered at the image resolution of 600 ×600 on a PC with an Intel 2.6 GHZ CPU, 3.0G RAM, and Nvidia Geforce 280 video card (1.0G video memory). For each dataset, preprocessing including fiber tracking and filtering, generating the guidance-vector field, and synthesizing a 2563 (5123) solid texture, costs 50 minutes, 2 minutes, and 10 minutes (50 minutes) respectively. The performances for visualizing datasets at the resolutions of 2563 and 5123 are 12 fps and 3 fps respectively.

There are many parameters involved in our pipeline. However, only some of these parameters are normally adjusted by the user. The other system parameters are set to stably generate reasonable results like the r-norm in Equation 4 and the number of pre-computed directions (set to be 60) used for the vector field guided solid texture synthesis Table 1 lists important parameters. To ease user interaction, one or multiple default values are predetermined.

Table 1
TABLE 1 Important parameters in the algorithm pipeline

6.2 The Cardiac Muscle Examples

3D DTI has previously been used to image cardiac muscle [28]. This method has proven capable of generating images showing cardiac fiber structure. Here, we examine the ability of 3D DTI to detect infarcted myocardium by analyzing the disruption in fiber structure 48 hours post infarct. 3D DTI was in fact able to detect infarcted myocardium and may hold promise as a noninvasive imaging modality requiring no radioisotopes, contrast agents, or radiation to assess myocardial infarct size and disruption of cardiac fiber structure. In conjunction with other imaging modalities, it may also hold promise in identifying viable myocardium by identifying areas of preserved cardiac fiber structure but abnormal wall motion.

A pig was anesthetized and underwent an echocardiographic examination. Left ventricular structure and function were confirmed to be grossly normal. A myocardial infarction was then induced in the pig by inserting a balloon-tipped catheter in a branch of one of the coronary arteries and inflating the balloon for 60 minutes to occlude flow. Two days following the catheter inflation procedure, the pig underwent a repeat echocardiographic examination. An area of hypokinesis was noted in the posterior wall of the left ventricle. The pig was then euthanized. The heart was removed and placed in cardioplegia solution at 4°C and taken to the MRI machine for the 3D DTI scan. Gross examination of the heart showed several areas of discoloration indicative of ischemic injury including one area in the posterior wall of the left ventricle. Histological analysis of this region of the heart confirmed the presence of a myocardial infarction (see Figure 7).

Figure 7
Fig. 7. Histology of the tissue samples in the native region (a) and the infarcted region (b). (b) confirms the myocardial infarction in the heart.

A GE 3T MRI scanner was used for imaging. A data volume of 128 ×128 ×34 was acquired with a voxel size of 1.86 ×1.86 ×2.4 mm. This data was then scaled to a volume of 256 ×256 ×180 with a voxel size of 0.93 ×0.93 ×0.93 mm. DTI fibers were generated by uniformly seeding in the data volume with a high seeding density (1/0.93 mm3 and jittering in order to cover the entire data volume without aliasing artifacts. The DTI fibers were then culled to remove those that were too similar to others [44].

Figure 6 shows the geometric rendering and volume illustration of a normal pig heart. Possible visual clutter due to high fiber density may appear from geometric rendering. Volume illustration can reveal hidden structure and the internal appearance by means of appropriate transparency assignment or cutting plane viewing. For instance Figure 8 illustrates the same model with interactive cutting through the z axis (from apex to atrium). Several different layers of muscle fibers could be seen through the left ventricular wall as seen on earlier histo-logical analysis (Figure 7).

Figure 8
Fig. 8. The layered slicing view for a normal pig heart data from the apex (a) toward the atrium (d). The weight w is set to be 0.2.
Figure 9
Fig. 9. Comparison of an abnormal pig heart and a normal pig heart. (a) A photograph of the abnormal pig heart. (b-c) The entire view of the abnormal and normal pig heart datasets, respectively. (d-e) The layered slicing view (another viewpoint) of the abnormal and normal hearts, respectively. For (b-e), the weight w is set to be 0.3. The differences are indicated by yellow circles.

The volume illustration provided the opportunity to view multiple cross-sectional images, which allows for an accurate determination of the size and location of the infarct. The induced myocardial infarction can be seen when compared to similar images from a normal heart (Figure 9). In Figure 9 (b) and (c), two datasets show different fiber density, which would usually indicate some abnormality of the dataset shown in (b). The abnormality can be further examined by viewing cross-sectional images as shown in (d) and (e). An area of the left ventricle is absent (yellow circle) in (d) corresponding to the histologically proven infarct zone. In contrast, the corresponding region of the normal heart is visualized in (e).

6.3 The Skeletal Muscle Example

We tested our method on skeletal muscles. We have scanned a foreleg and a hind leg from a sacrificed pig with a voxel size of 1.86×1.86×6 mm and a data volume of 128 × 128 × 34. The pig was positioned so that its legs were included in the field-of-view. A GE diffusion tensor imaging protocol was used, with two b values (0, 1000) in 25 directions. The data were then scaled to a volume of 256×256 × 180 with a voxel size of 0 .93 × 0 .93 × 0 .93 mm.

DTI fibers were generated by uniformly seeding in the data volume with a high seeding density (1 /0 .99 mm3) and jittering in order to cover the entire data volume without aliasing artifacts. The minimum linear anisotropy was set to 0.05. The DTI fibers were then culled to remove those that were too similar to others. The culling distance was set to 1.0 mm. The minimum culling distance was set to 0.5 mm.

Figure 10 shows the visualizations of a pig hind leg. Solely employing a solid texture (Figure 10 (c)) presents some structural patterns yet yields dim effects because no lighting is applied. Among all three results achieved by means of different blending weights, the one (b) that mixes the solid texture and the volume illumination effects provides a better insight to the internal muscle fiber orientation of the leg.

Figure 10
Fig. 10. Results for a pig leg dataset. From left to right, the weight w is 0.0, 0.3, and 1.0 respectively.

6.4 Comparison and Discussion

We have implemented and tested other visualization approaches in the same platform Figure 11 gives an example of the comparisons among the glyph representation [40] (a), the illuminated line rendering [27] (b), the colored streamline rendering [45] (c), the streamtube rendering [44] (d), the Hsu approach [20] (e), and our approach (f). All of them were accelerated with GPU Table 2 lists the performance statistics for the selected methods.

Figure 11
Fig. 11. Results for an abnormal pig heart dataset by means of the glyph representation [40] (a), the illuminated line rendering [27] (b), the colored streamline rendering where the principal eigenvector is mapped to RGB color channels [45] (c), the streamtube rendering [44] (d), the Hsu approach [20] (e), and our approach (f) with the weight w = 0.4.

Comparison with geometric rendering The glyph rendering can show the individual local tensor information well but can not effectively depict the connectivity of muscle fibers. The illuminated fibers show the orientation of the muscle fibers just as the geometric fiber rendering does, but the volume illustration provides a more complete depiction of the muscle appearance by simulating the internal appearance. In addition, possible visual clutter due to high fiber density may appear in the geometric rendering with the glyph, streamline, and streamtube representations. Moreover, the volume illustration could be modulated to visualize other properties like the fractional anisotropy [8].

Table 2
TABLE 2 Performance statistics in FPS with respect to the fiber density

The rendering performance of volume illustration is approximately proportional to the volume resolution regardless of the fiber density, while the performance of the geometric rendering is approximately proportional to the fiber density.

Comparison with texture-based methods The LIC-based approach [20] is designed to visualize the first and second vector fields. In contrast, our solid texture synthesis approach is suitable for simulating the muscle appearance to provide contextual illustration. Although the LIC-based texture can be displayed with volume rendering techniques, it is not clear how effective the LIC-based method is in 3D.

Compared to the work presented in Dong and Clapworthy [14], our approach shows two distinct differences. On one hand, we employ DTI images to compute the fiber orientations. The fiber orientation information provided by DTI images cannot be retrieved from CT or Visible Human datasets used in Dong and his colleagues [14]. On the other hand, our solid texture synthesis algorithm requires only a 2D example, making the construction more flexible and easier to use than a 3D volume sample as suggested by Dong and his colleagues [14] Discussion Our approach is different from standard DTI fiber visualization algorithms [42], [45]. Typically, DTI fiber visualization aims at reproducing the fiber structures by strictly obeying the fiber distributions extracted from the DTI images. Volume illustration of muscle requires us to incorporate certain aesthetics by means of solid texture synthesis and shape volume generation. Furthermore, the appearance of the rendered objects with volume illustration can be copied from 2D examples, thereby providing opportunities for 3D muscle visualization to achieve the effects of the hand-drawn illustrations.

SECTION 7

Conclusion

In this paper we have introduced a novel approach for illustrating volumetric muscle models that are generated from DTI images and 2D examples. With our system, we demonstrate how to construct fibrous structures and how to produce an expressive visualization of the muscle. The experiments show that our approach compares favorably with existing alternatives. We believe that our approach is an effective complement to previous geometry- or biomechanics-based muscle simulation methods.

Acknowledgments

We would like to thank the anonymous reviewers for the detailed and constructive comments. This work was initialized when Wei Chen was visiting Purdue University. Part of the implementation was built on the IVIS volume illustration system developed by Purdue university. This work is partially supported by 973 program of China (2010CB732504), NSF of China (No.60873123), the Research Initiation Program at Mississippi State University, MAFES Strategic Research Initiative Funding (CRESS MIS-741110), the U.S. DOE DE-FG02-06ER25733 and NSF 0633150, the U.S. NSF under Grant 0328984 and by the U.S. Department of Homeland Security Regional Visualization and Analytics Center (RVAC) Center of Excellence.

Footnotes

Wei Chen, Zhicheng Yan and Zi'ang Ding are with the State Key Lab of CAD&CG, Zhejiang University, E-mail: chenwei@cad.zju.edu.cn, yanzhicheng@cad.zju.edu.cn, dingziang@cad.zju.edu.cn.

Song Zhang is with the Department of Computer Science and Engineering, Mississippi State University, E-mail: szhang@cse.msstate.edu.

John Allen Crow, Ronald M. McLaughlin, Katie B. Mullins and Robert Cooper are with the College of Veterinary Medicine, Mississippi State University, E-mail: crow@cvm.msstate.edu, mcLaughlin@cvm.msstate.edu, Mullins@cvm.msstate.edu, rcooper@cvm.msstate.edu.

David S. Ebert is with the School of Electrical and Computer Engineering, Purdue University, E-mail: ebertd@ecn.purdue.edu.

Jun Liao is with Department of Agriculture and Biomedical Engineering, Mississippi State University, E-mail: jliao@abe.msstate.edu.

The first three authors contribute equally to this work.

Manuscript received 31 March 2009; accepted 27 July 2009; posted online 11 October 2009; mailed on 5 October 2009.

For information on obtaining reprints of this article, please send email to: tvcg@computer.org.

References

1. Restoration of DWI data using a Rician LMMSE estimator.

S. Aja-Fernandez, M. Niethammer, M. Kubicki, M. Shenton and C. F. Westin

IEEE Transactions on Medical Imaging, 27 (10): 1389–403, 2008.

2. Illumination in diverse codimensions.

D. C. Banks

In Proceedings of ACM SIGGRAPH, pages 327–334, 1994.

3. In vivo fiber tractography using DT-MRI data.

P. J. Basser, S. Pajevic, C. Pierpaoli, J. Duda and A. Aldroubi

Magnetic Resonance in Medicine, 44: 625–632, 2000.

4. Rician noise removal in diffusion tensor MRI.

S. Basu, P. Fletcher and R. Whitaker

Lecture Notes in Computer Science, 4190: 117, 2006.

5. Three-dimensional representation of complex muscle architectures and geometries.

S. Blemker and S. Delp

Annals of Biomedical Engineering, 33: 661–C673, 2005.

6. VolumeShop: An interactive system for direct volume illustration.

S. Bruckner and M. E. Gröller

In Proceedings of IEEE Visualization, pages 671–678, Minneapolis, USA, October 2005.

7. Imaging vector fields using line integral convolution.

B. Cabral and L. Leedom

In Proceedings of ACM SIGGRAPH, pages 263–270, 1993.

8. Abstractive representation and exploration of hierarchically clustered diffusion tensor fiber tracts.

W. Chen, S. Zhang, S. Correia and D. S. Ebert

Computer Graphics Forum (Special Issue of Eurographics/IEEEVGTC Symposium on Visualization), 27 (3), 2008.

9. Texture evolution: 3D texture synthesis from single 2D growable texture pattern.

Y. Chen and H. IP

The Visual Computer, 20 (10): 650–664, 2004.

10. Feature-aligned volume manipulation for illustration and visualization.

C. Correa, D. Silver and M. Chen

IEEE Transactions on Visualization and Computer Graphics, 12 (5): 1069–1076, 2006.

11. Diffusion tensor magnetic resonance image regularization.

O. Coulon, D. Alexander and S. Arridge

Medical Image Analysis, 8 (1): 47–67, 2004.

12. Comparison of fiber tracts derived from invivo DTI tractography with 3D histological neural tract tracer reconstruction on a macaque brain.

J. Dauguet, S. Peled, V. Berezovskii, T. Delzescaux, S. K. Warfield, R. Born and C.-F. Westin

NeuroImage, 37: 530–538, 2007.

13. An anatomy-based approach to human muscle modeling and deformation.

F. Dong, G. Clapworthy, M. Krokos and J. Yao

IEEE Transactions on Visualization and Computer Graphics, 08 (2): 154–170, 2002.

14. Volumetric texture synthesis for nonphotorealistic volume rendering of medical data.

F. Dong and G. J. Clapworthy

The Visual Computer, 21 (7): 463–473, 2005.

15. Volume illustration: Non-photorealistic rendering of volume models.

D. Ebert and P. Rheingans

In Proceedings of IEEE Visualization, pages 253–264, Boston, USA, October 2002.

16. Image quilting for texture synthesis and transfer.

A. Efros and W. T. Freeman

In Proceedings of ACM SIGGRAPH, pages 341–346, 2001.

17. Visualization of white matter tracts with wrapped streamlines.

F. Enders, N. Sauber, D. Merhof, P. Hastreiter, C. Nimsky and M. Stamminger

In Proceedings of IEEE Visualization 2005, pages 51–58, 2005.

18. DTI visualization of tensor fields using superquadric glyphs.

D. B. Ennis, G. Kindlman, I. Rodriguez, P. A. Helm and E. R. McVeigh

Magnetic Resonance in Medicine, 53 (1): 169–176, 2005.

19. Pyramid-based texture analysis and synthesis.

D. J. Heeger and J. R. Bergen

In Proceedings of ACM SIGGRAPH, pages 229–238, 1995.

20. Generalized line integral convolution rendering of diffusion tensor fields.

E. Hsu

In Proceedings of International Society of Magnetic Resonance Medical, page 790, 2001.

21. Stereological techniques for solid textures.

R. Jagnow, J. Dorsey and H. Rushmeier

ACM Transactions on Graphics, 23 (3): 329–335, 2004.

22. Strategies for direct volume rendering of diffusion tensor fields.

G. Kindlmann, D. Weinstein and D. Hart

IEEE Transactions on Visualization and Computer Graphics, 6 (2): 124–138, April-June 2000.

23. Diffusion tensor visualization with glyph packing.

G. Kindlmann and C.-F. Westin

IEEE Transactions on Visualization and Computer Graphics, 12 (5): 1329–1336, 2006.

24. Solid texture synthesis from 2D exemplars.

J. Kopf, C.-W. Fu, D. Cohen-Or, O. Deussen, D. Lischinski and T.-T. Wong

ACM Transactions on Graphics, 26 (3), 2007.

25. Example-based volume illustrations.

A. Lu and D. Ebert

In Proceedings of IEEE Visualization, pages 655–662, Minneapolis, USA, October 2005.

26. Volume illustration: Designing 3D models with internal textures.

S. Owada, F. Nielsen, M. Okabe and T. Igarashi

In Proceedings of ACM SIGGRAPH, pages 322–328, San Diego, USA, August 2004.

27. Visualization of the fibrous structure of the heart.

T. Peeters, A. Vilanova, G. Strijkers and B. H. Romeny

In Proceedings of Vision Modeling and Visualization, pages 309–317, 2006.

28. Towards a statistical atlas of cardiac fiber structure.

J.-M. Peyrat, M. Sermesant, X. Pennec, H. Delingette, C. Xu, E. McVeigh and N. Ayache

In MICCAI, pages 297–304, 2006.

29. Reconstruction and visualization of fiber and laminar structure in the normal human heart from ex vivo diffusion tensor magnetic resonance imaging (DTMRI) data.

D. Rohmer, A. Sitek and G. T. Gullberg

Investigative Radiology, 42 (11): 777–789, 2007.

30. Anatomy-based modeling of the human musculature.

F. Scheepers, R. E. Parent, W. E. Carlson and S. F. May

In Proceedings of ACM SIGGRAPH, pages 163–172, 1997.

31. Histological validation of myocardial microstructure obtained from diffusion tensor magnetic resonance imaging.

D. Scollan, A. Holmes, R. Winslow and J. Forder

American Journal of Physiology, 275: 2308–2318, 1998.

32. In vivo diffusion tensor imaging of the human calf muscle.

S. Sinha, U. Sinha and V. R. Edgerton

Journal of Magnetic Resonance Imaging, 24 (1), 2006.

33. Fast robust automated brain extraction.

S. M. Smith

Human Brain Mapping, 17 (3): 143–155, 2002.

34. Sobotta Atlas of human anatomy

J. Staubesand and A. N. Taylor

Urban and Schwarzenberg Baltimore-Munich, 1990.

35. Illustration motifs for effective medical volume illustration.

N. Svakhine, D. Ebert and D. Stredney

IEEE Computer Graphics and Applications, 25 (3), 2005.

36. Lapped solid textures: Filling a model with anisotropic textures.

K. Takayama, M. Okabe, T. Ijiri and T. Igarashi

ACM Transactions on Graphics, 27 (3), 2008.

37. Creating and simulating skeletal muscle from the visible human data set.

J. Teran, E. Sifakis, S. Blemker, V. Ng-Thow-Hing, C. Lau and R. Fedkiw

IEEE Transactions on Visualization and Computer Graphics, 11 (3): 317–328, 2005.

38. Variational frameworks for DT-MRI estimation, regularization and visualization.

D. Tschumperle and R. Deriche

In Proceedings of IEEE International Conference on Computer Vision, pages 116–121, 2003.

39. DTI visualization with stream surfaces and evenly-spaced volume seeding.

A. Vilanova, G. Berenschot and C. van Pul

In Eurographics/IEEE Symposium on Visualization, pages 173–182, 2004.

40. An introduction to visualization of diffusion tensor imaging and its applications.

A. Vilanova, S. Zhang, G. Kindlmann and D. H. Laidlaw

In Visualization and Image Processing of Tensor Fields. Springer, 2005.

41. Texture synthesis by fixed neighborhood searching

L.-Y. Wei

PhD thesis, Stanford University, 2002.

42. Interactive volume rendering of thin thread structures within multivalued scientific datasets.

A. Wenger, D. Keefe, S. Zhang and D. H. Laidlaw

IEEE Transactions on Visualization and Computer Graphics, 10 (6): 664–672, November 2004.

43. Geometrical diffusion measures for MRI from tensor basis analysis.

C. Westin, S. Peled, H. Gubjartsson, R. Kikinis and F. Jolesz

In Proceedings of International Society Magnetic Resonance in Medicine (ISMRM), 1997.

44. Visualizing diffusion tensor MR images using streamtubes and streamsurfaces.

S. Zhang, C. Demiralp and D. H. Laidlaw

IEEE Transactions on Visualization and Computer Graphics, 9 (4): 454–462, October 2003.

45. Heart-muscle fiber reconstruction from Diffusion Tensor MRI.

L. Zhukov and A. H. Barr

In Proceedings of IEEE Visualization, pages 79–84, 2003.

Authors

No Photo Available

Wei Chen

Member, IEEE

No Bio Available
No Photo Available

Zhicheng Yan

Student Member, IEEE

No Bio Available
No Photo Available

Song Zhang

Member, IEEE

No Bio Available
No Photo Available

John Allen Crow

No Bio Available
No Photo Available

David S. Ebert

Fellow, IEEE

No Bio Available
No Photo Available

Ronald M. McLaughlin

No Bio Available
No Photo Available

Katie B. Mullins

No Bio Available
No Photo Available

Robert Cooper

No Bio Available
No Photo Available

Zi'ang Ding

No Bio Available
No Photo Available

Jun Liao

No Bio Available

Cited By

No Citations Available

Keywords

IEEE Keywords

No Keywords Available

INSPEC: Controlled Indexing

image texture, medical image processing, tensors, vectors.

More Keywords

No Keywords Available

Corrections

No Corrections

Media

No Content Available

Indexed by Inspec

© Copyright 2011 IEEE – All Rights Reserved