• Abstract

# A Novel Interface for Interactive Exploration of DTI Fibers

Visual exploration is essential to the visualization and analysis of densely sampled 3D DTI fibers in biological speciments, due to the high geometric, spatial, and anatomical complexity of fiber tracts. Previous methods for DTI fiber visualization use zooming, color-mapping, selection, and abstraction to deliver the characteristics of the fibers. However, these schemes mainly focus on the optimization of visualization in the 3D space where cluttering and occlusion make grasping even a few thousand fibers difficult. This paper introduces a novel interaction method that augments the 3D visualization with a 2D representation containing a low-dimensional embedding of the DTI fibers. This embedding preserves the relationship between the fibers and removes the visual clutter that is inherent in 3D renderings of the fibers. This new interface allows the user to manipulate the DTI fibers as both 3D curves and 2D embedded points and easily compare or validate his or her results in both domains. The implementation of the framework is GPU-based to achieve real-time interaction. The framework was applied to several tasks, and the results show that our method reduces the user's workload in recognizing 3D DTI fibers and permits quick and accurate DTI fiber selection.

SECTION 1

## Introduction

Diffusion tensor imaging (DTI) is a non-invasive magnetic resonance imaging (MRI) technique that measures the speed and direction of water diffusion in biological tissues. The characteristics of water diffusion in a biological structure (e.g., heart) can be mathematically summarized by a diffusion tensor field. By tracking the trajectories of the fastest diffusion in a diffusion tensor field with streamline methods [2], a DTI dataset can be represented with a set of fiber tracts, or three-dimensional pathways. This process, called tractography, shows the connectivity and distribution of the fibers and has been widely used in visualization and analysis of DTI datasets [15], [18], [27].

Exploring and analyzing fiber tracts in the three-dimensional space is challenging due to the visual clutter caused by the complexity of the geometry. For example, a fiber model for white matter may contain more than ten thousand fibers, making it difficult to derive useful insights from the dataset. A number of fiber clustering techniques [5, 18, 20, 24, 26] have been used to group similar fibers (i.e., fibers that lie close to one another and follow similar trajectories through the tensor field) into automatically representative fiber bundles. Graphical representations of such fiber bundles reduce the visual complexity of the dataset thereby facilitating a user's exploration of the data and allow him or her to more quickly gain insights into the structural integrity and connectivity of the fibers [6], [23]. However, renderings of clustered bundles can still suffer from occlusion (i.e., one cluster obscuring another) that impedes the perception of the DTI fibers or the selection of specific fibers [3] for further analysis.

The ability for scientists to interactively explore and select DTI fibers for inspection or for use in statistical analysis is critical for DTI research. The difficulty of interacting with DTI fiber models highlights the need for better visual representations and more convenient user interfaces. Existing solutions attempt to solve the problem by employing new visual forms or novel interaction methods such as interactive selection schemes [1], [3], dynamic query [23], geometric simplification [6], [10], color-mapping [9], [8], texture patterning of fiber dissimilarity [13], and uncertainty visualization [11]. These methods have greatly improved 3D DTI fiber visualization. Nevertheless, most of them operate in the three-dimensional space, where the geometry is often occluded after being projected to a 2D viewport. To achieve satisfactory results, a user must be trained to perform careful and efficient exploration of the fiber model and to make anatomically correct fiber tract selections: a learning curve that can be steep and time consuming.

The current situation in visualizing complex DTI fiber models is analogous to the difficulty in exploring and visualizing volume datasets. Transfer function design and statistical analysis tools [25] have revolutionized volume rendering by facilitating user interactions and promoting useful insights about the data. The reason behind the success of the transfer functions in volume rendering includes the clarity and simplicity of a 2D representation and its direct coupling with 3D representation. However, to our knowledge, a similar interaction method has not been applied to DTI fiber tracts.

In this paper, we describe a novel DTI fiber exploration scheme that builds upon recent methods for DTI data analysis and fiber exploration techniques. The main contributions include:

• An effective interaction mode that combines 3D, 2D, and statistical views to broaden the user's exploratory space;

• A coherent local-dimensional embedding algorithm that preserves the spatial relationships of the fiber tracts and provides the user with an uncluttered 2D representation of the data; and

• A set of visualization, manipulation, and statistical analysis tools that reduces the user time and mental workload in recognizing 3D DTI fibers.

Figure 1 shows a screen-shot of the main interface of our system. Our integrated system is GPU-accelerated and achieves interactive performance. The rest of this paper is organized as follows. Section 2 provides a brief summary of related work; Section 3 and Section 4 describe the details of our method and its implementation; Section 5 demonstrates the efficiency of the new interface with several case studies and a user evaluation; and Section 6 presents our conclusions.

Fig. 1. A snapshot of our DTI fiber exploration system. The figure shows a sagittal view fibers in the corpus callosum and cingulate bundle (left-right orientation) of a human brain dataset (anterior is to the viewer's left).
SECTION 2

## Related Work

DTI fibers are usually integrated along the longest eigenvectors in a tensor field [2]. They can be represented and displayed as streamlines [15], streamtubes and streamsurfaces [27]. The geometric shape of a set of fibers can be further simplified to a more abstractive visual form such as wrapped streamlines [10] or topological simplification [22] or by using hierarchical principal curves [6]. A novel set of interaction techniques introduced in Sherbondy et al. [23] allows for exploring brain connectivity and interpreting pathways. In [23], the key operation offered to neuroscientists is the placement and manipulation of box-shaped or ellipsoid-shaped regions in coordination with a simple and flexible query language. A similar selection scheme is presented in Blaas et al. [3]. A clinical study shows that this approach is highly reproducible for fractional anisotropy (FA) calculated over the selected fiber bundles. To visually differentiate pathways, four different pattern styles are used to encode the local dissimilarity information of fibers, yielding an online navigation tool for fiber connectivity [13]. The dissimilarity within a group of fibers may be illustrated in cross section with an appropriate coloring scheme [9], [8], making it easy to discern a cluster by color. Other features derived from the fibers could also be identified and visualized, such as the uncertainty arising from noise and partial volume effects [11]. Most of these schemes focus on the representation of DTI fibers in 3D space, limiting the amount of information that can be shown and explored before excessive visual clutter occurs. A recent work [14] proposes to link an embedding in the plane and a hierarchical clustering tree with the 3D fiber tracts, facilitating navigation through complex fiber tracts.

One application for our approach is DTI fiber clustering. By measuring the proximity among fibers, a fiber set can be grouped into anatomically meaningful fiber bundles. Different proximity measures have been proposed such as the single-linkage hierarchical clustering method. This method is based on the mean distance [26] and yields excellent results in practice [18]. Various clustering schemes can be chosen. Unsupervised partitions can be created by leveraging graph theory like the normalized cut [5]. In addition, a manifold learning scheme [24] can be used to construct proximity measures that capture the neighborhood structures in the high-dimensional data space. Supervised clustering is an alternative choice. Possible approaches include the identification of regions of interest (ROIs), fine-tuning of clustering parameters, or direct manipulations of fiber tracts [3], [23]. To reduce user interaction time, a clustered atlas model can be employed to rapidly construct a set of clusters for a new dataset. The clusters in the atlas model are augmented with expert anatomical labels and are transferred to new models by spectral learning [20] or affine registration [16].

SECTION 3

## Method

The key idea is that a useful coordination between a simplified feature space view of fibers and the exact geometry in a standard 3D view and the interaction between them is essential for exploring DTI fibers. Many approaches have employed manifold learning algorithms to define a low-dimensional embedding of the fiber tracts that preserves the neighborhood structures in the high-dimensional space [24]. These embedding schemes have been mostly applied to DTI fiber clustering [20]. However, the clarity and efficiency of these techniques have rarely been exploited in other aspects of DTI fiber study such as user-drive interactive fiber tract selection. Our method makes use of both the simplicity of the low-dimensional embedding and the fidelity of the 3D fibers to enhance interaction, visualization, and exploration.

### 3.1 The Interface

The layout shown in Figure 1 is designed to enable efficient browsing, manipulation, and quantitative analysis of DTI fiber tracts. It consists of two main components. On the left side, a 3D view displays the fiber tracts and provides interactions such as rotation, lens viewing, coloring, slicing, and selection. On the right side, a 2D embedding of the DTI fibers is shown. The interface also contains several views that enable interactive filtering and numerical analysis.

### 3.2 Proximity-preserving Two-dimensional Embedding

Our idea is inspired by multi-dimensional scaling (MDS) techniques [12] that are designed to provide a visual representation of the pattern of proximities. Given a set of points in a high-dimensional space, MDS aims to compute another set of points X in a low-dimensional (two or three) space such that the point distances in the high-dimensional space are preserved as much as possible.

With a set of fibers F = {fi i = 1 ,2 ,3 n} that can be regarded as points in a high-dimensional space, a proximity measure produces a distance matrix G = { δij i j = 1 ,2 ,3 n},of which δij is the distance between fi and fj. Using the standard MDS algorithm [4], one can compute a set of two-dimensional points X = {pi =(xi yi) ∊ R2 i = 1,2,3 n} by minimizing a badness-of-fit measure (called raw stress): TeX Source $$\sigma _r (X) = \sum\limits_{i < j} {(d_{ij} - \delta _{ij})^2 }$$ where dij denotes the Euclidean distance between pi and pj, and pi and pj are the corresponding points of fi and fj respectively.

Fig. 2. (a) A pig heart dataset with 1,725 fiber tracts; (b) The two-dimensional MDS representation of (a) with respect to the mean distance [27]. The color (smooth transition from (1,1,0) to (1,0,1)) and size of each point are encoded to be proportional to the fractional anisotropy and the length of its corresponding fiber tract respectively.

The designed stress can be minimized by the steepest descent algorithms or iterative majorization approaches [4]. We chose one of the latter approaches called "Scaling by Majorizing a Complicated Function" (SMACOF). For details, please refer to [4].

The point set X = {pi = (xi yi)} computed by minimizing Equation 1 has an one-to-one correspondence to the input fiber model F = {fi i = 1,2,3 ... n}. Drawing these points constructs a 2D embedding of the fiber set, as shown in Figure 2 (a-b). Each point in the 2D plane corresponds exactly to a fiber. The properties of each point such as the size and color can be mapped to the properties of its corresponding fiber, such as the average fractional anisotropy, the average relative anisotropy, the length, the average curvature, or the clustering membership. Using MDS yields the following benefits:

• The proximity between any pair of fiber tracts is ideally identical or close to the one between their counterparts in the 2D space.

• The fiber properties can be encoded with the size, color, glyph, or texture pattern of the points.

• Multiple feature vectors and distance matrices can be used in MDS. This flexibility expands the possibility of data exploration.

• It is easier to view the 2D embedding of the fiber tracts and explore their relationships in a single 2D plane compared to the fiber exploration in the 3D space.

Table 1 lists selected proximity measures provided in our framework. Specifically, the user is allowed to adjust the weights α β, and γ for determining dW(Q,R Figure 3 shows the MDS representations of a brain dataset under different proximity measures. They exhibit similar distributions but have small differences in regions, for example the bottom left-hand corner. In the last two rows of Table 1, characteristics of DTI fibers such as the fractional anisotropy, the length, the discrete curvature, and the linear anisotropy can be incorporated into the 2D embedding by modifying the distance term with respect to the chosen proximity measure (see Equation 1). Depending on applications, more properties from DTI data can be employed.

TABLE 1 Examples of proximity measures for two fiber tracts Q and R. λ1, λ2 and λ3 are three real eigenvalues of a diffusion tensor, and . FA, RA, and AC denote the averages of the fractional anisotropy, the relative anisotropy, and the discrete curvature along the fiber respectively. The three parameters α β and γ can be interactively adjusted by the user.
Fig. 3. The MDS representations of a pig heart dataset with various proximity measures. (a) The minimum distance; (b) Weighted with α = 0.2 β = 0.8 and γ = 0.0; (c Weighted with α = 0.6 β = 0.0 and γ = 0.4 ; (d) Weighted with α = 0.0 β = 0.3 and γ = 0.7 .To be consistent with (a), (b-d) are generated with the consistent MDS representation scheme, easing the recognition of their differences.

### 3.3 Consistent Two-dimensional Representations

Hierarchical fiber clusters and principal fibers [6], [10] are effective in representing the structure of the DTI fibers in multiple levels of detail. The exploration of hierarchical principal fibers could be assisted by the 2D DTI fiber embedding. Although different levels share some common points and the point distributions appear to have similar patterns, the point locations can be inconsistent (see Figure 4 (a-c)) because the MDS algorithm randomly sets an initial value for each low-dimensional point. In addition, the atlas-based manipulation suffers from the same problem, i.e., two datasets may have quite different point layouts, resulting in inconsistent representations.

Fig. 4. (a) A fiber cluster (left) and its principal curve (right) in a pig leg model; (b) The MDS representations of the model, where the points indicated by a yellow circle correspond to the cluster shown in (a); (c) Applying the MDS representation to a model that is simplified from the input model. Notice that (b) and (c) show distinctive layouts. The purple point indicated by the yellow circle is the embedding of the simplified curve in (a); (d) Our consistent scheme yields an aligned layout with respect to (b) for two models.
Fig. 5. Interactive two-dimensional motifs demonstrated with a pig leg dataset. (a) A lens viewing window is shown in the top right corner to depict the details in the user-specified region (the small rectangle); (b) Free selection with a closed stroke; (c) Applying K-means to the 2D points shown in (a); (d) Applying K-means to the 2D MDS of a hierarchical simplified version of the input dataset.

The inconsistency can be addressed by a simple scheme. Assume we have two fiber models F1 and F2, which have a logically common subset F* = {k|pk2F2, pk2pjk1F1 }. Here pk2pjk1 means that pk2 may not be identical to pjk1 but is logically close to it. One example is the principal fiber computed from a fiber cluster. It could be some fiber in the cluster but may also be a different one that is very close to some fiber. To have a coherent 2D MDS layout of F2 with respect to the 2D MDS layout X1 of F1, a new stress for computing X2 is: TeX Source $$\sigma _r ({\bf{X}}^2) = \sum\limits_{i < j} {(d_{ij} ({\bf{X}}^2) - \delta _{ij}^2 })^2 + \sum\limits_{p_k^2 \in {\bf{X}}^2, k \in F^* } {(p_k^2 - p_{jk}^1)^2 }$$

Solving Equation 2 is similar to that for Equation 1 Figure 4 (d) shows a corrected MDS layout with respect to Figure 4 (b-c).

### 3.4 Two-dimensional Interactions

The MDS-based 2D embedding is shown in a window on the right side of the interface, providing a clear and simple 2D view of the embedded points. Possible interactions in this 2D view include coloring, glyph shape mapping, and texture mapping, In addition, the following two interactions are helpful for interactive exploration (see Figure 5):

• Selection While DTI fiber clustering in the 3D space requires either drawing multiple regions of interest or individually picking the fibers, in the 2D view the embedded points can be simultaneously scanned by eye, allowing simple manipulation. For instance, the user can intuitively draw a closed curve to select a list of closely located embedded points inside the curve. The selection will be immediately highlighted in the 3D view.

• Clustering A good MDS representation preserves the proximity between the high-dimensional DTI fibers. Thus, DTI fiber clustering in the 2D embedded space might also yield meaningful results. The results can be visually inspected using the 2D space without visual clutter. Furthermore, the 2D clustering results can be immediately reflected in 3D, enabling detailed inspection in the 3D space.

### 3.5 Numerical Exploration Views

To allow for interactive exploration of derived characteristics of the DTI fibers, additional histogram windows are added. Possible input properties include the fiber length, the average linear anisotropy (LA), the average fractional anisotropy (FA), and the average curvature along each fiber tract. Clipping on the one-dimensional histograms filters out DTI fibers whose properties are not in the meaningful ranges. The selected fibers can be colored to highlight the result. The clipping can also be applied to the results from selection, clustering, or other manipulations in the 2D and 3D views to assist insights into underlying anatomy or pathological changes Figure 6 demonstrates a simple filtering process with a heart dataset.

### 3.6 Three-dimensional Interactions

To provide the flexibility of interactive exploration in the 3D space, easy selection operations are supported (see also Figure 7):

• Individual fiber specification A user can select a single fiber by mouse clicking and specify its membership to an underlying cluster.

• Free fiber selection A user can choose a list of fibers by drawing strokes onto the visualization of the 3D fibers.

• Multiple-box ROIs determination Three (or more) boxes can be interactively placed, sized, and translated, forming a multiple-box ROI. The fiber tracts passing through these boxes can be highlighted and chosen as a new bundle. The specification, scaling, and movement of these boxes can be conveniently manipulated with the interface, making the fiber selection convenient because more than one constraints on location are imposed onto the intended fiber bundles. Note that in some situations, a single box is adequate for choosing a bundle (see Figure 7 (c)), while multiple boxes are required in other cases (see Figure 7 (d)).

The 2D embedding of the DTI fibers is simple and clear but does not represent the shapes of the 3D fibers accurately. The 3D representation describes the shapes and distribution of fibers well, which is usually complicated and hard to manipulate. Either mode by itself may pose problems for DTI fiber perception and exploration. The combination of the 2D and 3D views in our interface facilitates fast and accurate DTI fiber analysis by providing multiple interaction means.

Fig. 6. Progressive filtering to a pig heart dataset. (a) 6,644 fiber tracts; (b) 3,506 fiber tracts after applying the length filter; (c) 2,218 fibers by choosing the ones whose FAs are larger than 0.1. (d) 592 fibers by culling the ones whose average curvature is smaller than 0.3.

Coupled Selection Fibers can be selected in the 3D space with spatial constraints, e.g., the multiple-box selection mode. Direct manipulation in the three-dimensional view is supported. Interactive selection could also be performed on the two-dimensional view or the numerical exploration views. Any selection operation in one view will evoke the visualization of selected objects in all views. This helps harness the human brain's ability for parallel processing and association in exploring the DTI fibers.

Cross Validation Many operations are coupled between the 2D and 3D views. Zooming, rotating, coloring, selecting, and abstracting techniques are present in both the two-dimensional and three-dimensional views. The operation in one view can be validated and corrected in other views. For instance, the 3D fiber clustering can be fine-tuned by identifying and removing outliers in either the 2D or 3D view. The user can freely switch between the views to achieve his or her goals and, at the same time, check the validity of the operation in other views. This will likely increase the time efficiency of the operation and decrease the probability of error.

Atlas-based Manipulation When many subjects are to be investigated, we can employ an atlas model to learn common structures present across subjects. Our specific interface for comparative visualization opens new opportunities to align, compare, and analyze multiple DTI fiber data.

In particular, our interface provides three clustering schemes that will be demonstrated in Section 5.

• The first mode solely performs automatic or semi-supervised clustering in the 3D space. This mode has been widely used in the DTI tractography community.

• The 2D embedding of DTI fibers allows for direct manipulation and clustering in the 2D plane, given that the 2D MDS configuration faithfully captures the characteristics of the fibers.

• One distinctive feature of our interface is the comprehensive depiction to the fiber model in multiple aspects. A new dual domain clustering mode is enabled by freely switching between views and combining compatible operations.

Fig. 7. 3D interactive selection. (a) Free selection; (b) A lens viewing window is used to depict the details in the small yellow rectangle; (c) A fiber bundle selected with a box; (d) A fiber bundle in purple is selected with three boxes. The red, yellow, and green boxes determine fibers that consist of the bundle in purple and the bundles in red, yellow, and green respectively. One box cannot accomplish this task.
SECTION 4

## Implementation

We implemented the user interface with Microsoft Visual C++ 2008 and OpenGL and tested it in a PC equipped with an Intel Core 2 Duo 2.4 GHz CPU, 2G host memory, and a Nvidia GTX 280 graphics card. In the interface, a single view is divided into multiple viewports to support multiple views in a single window. The fiber selection is achieved with the pick and selection features of OpenGL.

The rendering of DTI fibers utilized a GPU-accelerated illuminated line algorithm [17]. The visualization, manipulation, and navigation of DTI fibers are in real-time. The implementation of the semi-supervised clustering is adequately rapid for interactions: it takes only 1 second for a model with 4,000 fibers. However, computing the distance matrix and the MDS configuration is much more time consuming. Considering the data-parallel nature of matrix multiplications involved in these computations, a parallel acceleration scheme was employed by using the CUDA BLAS language [19]. The acceleration achieved for the construction of the distance matrix and the solving of MDS is more than 5 folds, as shown in Table 2.

TABLE 2 Time statistics in seconds with GPU and CPU

In addition to the tractography, other DT-MRI visualization schemes were incorporated into the 3D view of our interface, including the ellipsoidal glyphs [15], streamtube [27], and HyperLIC [28] representations. The streamtube representation has the same 2D embedding as the streamline representation, while the other two cannot.

SECTION 5

## Results and Discussions

Three co-authors including two neuropsychologists and one cardiologist have tested our system with a sequence of datasets: four human brain datasets, two pig heart datasets, and one pig leg model. The DTI resolutions for the these datasets are 1.7 mm × 1.7 mm × 1.7 mm for the brain datasets, 1.17 mm × 1.17 mm × 2.4 mm for the pig hearts, and 0.938 mm × 0.938 mm × 6 mm for the pig leg. The fiber numbers for the datasets are 13,644, 12,121, 13,169, 3,520 (for the four brain datasets), 6,644, 2,717 (for the two pig heart datasets), and 6,097 (for the pig leg dataset) respectively. After a short practice with a user manual on the software, each user individually conducted several case studies. In general, the users were satisfied with the interactivity and performance of the user interface.

### 5.1 Embedded DTI Fiber Clustering on a Pig Leg Dataset

For some models, the 2D embedding by MDS exhibits clear patterns and are suitable for rapid classification Figure 8 illustrates a clustering process of a pig leg dataset (Figure 8 (a)) with our interface. To investigate, the user first used the length and curvature filters to hide the extremely short curves and the curves with very high average curvatures (Figure 8 (b)). Then, a 2D embedding was produced. The user navigated the embedding fiber tracts on the 2D view and used the K-means algorithm to perform clustering in the 2D space (Figure 8 (c)). If the initial results were overly segmented, the user would then manually merge or correct the embedded fibers in the 2D plane to get a satisfactory result (Figure 8 (d)). The entire process including the MDS computation took about 20 minutes.

Fig. 8. A simple and effective clustering for a pig leg model

### 5.2 Dual Domain Clustering for a Pig Heart Model

Cardiac muscles are naturally grouped into layers and tracts [21]. Visualization of these layers and tracts could be helpful for a cardiologist examining the localized effects of various heart diseases. However, the heart model consists of many spatially close muscle fibers, the grouping of which is more challenging than the pig leg muscle. Using the 2D embedding process alone can hardly get a reasonable result. Therefore, more user interactions are needed. From an in-depth examination of the 3D view, the orientations and shapes of the various fiber bundles are significantly different, as shown in Figure 9 (a). This inspection in conjunction with the anatomical knowledge of the cardiologist induced a feasible solution for clustering the fiber tracts.

The entire process was divided into three stages. At the beginning, a length-based filtering was used to remove very short fiber tracts, most of which were likely the result of noise. Then, the user carefully studied possible types of fiber bundle structures by performing coupled query and selection. The user found that the curvature-based measure (i.e., let γ be 1.0 in the weighted proximity measure presented in Table 1) is effective in distinguishing fiber tracts and yielded four types of fiber bundles. The user then manually labeled some fiber tracts in potential fiber bundles. Subsequently, the user-specified labels were propagated to all other fibers by leveraging a semi-supervised learning approach [30]. Note that the mean distance [27] was used for the proximity measure at this stage Figure 9 shows the exploration process accomplished in 30 minutes. The analysis allowed for the tentative identification of a papillary muscle in one of the hearts examined. DTI analysis identified fiber tracts suggesting muscle fibers running from the apex toward the base of the heart along with the more abundant concentric muscle fibers.

Fig. 9. A dual domain clustering for (a) a pig heart model with 2,717 fibers; (b) Noisy data is removed by using the length filter; (c) Using the curvature filter, some fiber bundles are clearly formed; (d) The MDS with respect to (c); (e) Manual labeling on some potential fiber bundles, which can be used as the input for a semi-supervised clustering process; (f) 13 clusters were generated after refinement.

### 5.3 Atlas-based Clustering for Brain Models

Comparative visualization and exploration are essential for cross validation and decision making. Exploring brain models is indeed a time-consuming procedure of trial and error for clinical researchers. In addition to the interactive exploration, we designed a comparative interface that allows two models to share a common workspace enabling comparative analysis with other methods Figure 10 illustrates an atlas-based clustering pipeline, which is explained below.

This study began with a clustered (abbr. A) and an unclustered (abbr. B) brain model. Models were scanned and processed with the same set of settings and parameters. The main steps were sequentially performed as follows: (a) An abstractive representation for A with the principal fiber algorithm was computed [6]; (b) the two models were interactively aligned with a set of transformation widgets; (c) short fibers in B were removed with the length filter; (d) B was merged with the principal fibers of A, yielding B*; (e) the added fibers from A were regarded as user-specified labeling information; (f B* was then clustered by means of a semi-supervised learning algorithm [30]; (g) unusual results were culled using the coupled query; (h) results were refined interactively. The process took an experienced neuropsychologist 40 minutes, and the result was appropriate for further analysis.

Fig. 10. The pipeline of an atlas-based clustering

### 5.4 Expert Evaluation

A preliminary user test was conducted to evaluate the capability of our interface for improving the efficiency of cerebral white matter tract selection. The goal was to identify well-known fiber tracts and test for the efficiency of tract selection and tract refinement (e.g., the ease with which erroneous fibers are identified on visual inspection and removed or the ease with which inadvertently removed fibers can be added back). This initial test focused on the basic manipulation tools (e.g., brush tool, single-fiber selection tool) and box tools with an emphasis on the utility of the novel 2D MDS. The evaluation was performed by two clinical researchers with considerable knowledge of white matter anatomy and experience with other tract selection platforms. The fiber model was generated from a healthy elderly subject who is part of a research study database. The research study was approved by the Institutional Review Board at Butler Hospital, Providence, RI, and the participant provided written informed consent.

Fiber tracts selected by the clinical researchers for interface evaluation included the corpus callosum, bilateral cingulate bundles, right superior longitudinal fasciculus, right uncinate fasciculus, and bilateral corticospinal tract (superior to the pyramidal decussation). They were selected because the users are familiar to them and they represent a combination of commissural, association, and projection fibers. Moreover, the users deliberately chose some tracts that are relatively easy to select because their fibers are generally oriented mainly in a single plane (e.g., anterior-posterior in the case of the cingulate bundles) and others with trajectories that pass through several planes (e.g., uncinate fasciculus). The users spent approximately 30 minutes gaining a familiarity with the software.

The specific tasks performed by the users to identify tracts varied by user and tract but the broad operations employed were similar across tracts, and involved initial selection, refinement, and classification. The users first inspected a fiber model interactively by rotating it and then decided on an optimal starting orientation for selecting a particular target tract. Then, specific tools were used depending on the trajectory of the tract and the user's preference. In general, the users found the brush tool to be very helpful for initial tract selection and removing unwanted fibers particularly when the unwanted fibers had a similar orientation. The box tool was helpful for making initial selections of curved tracts (e.g., uncinate fasciculus).

At first, the 2D MDS display seemed irrelevant to the process of tract selection. This initial impression was due to the lack of the awareness of the mapping of the spatial arrangement of the dots in 2D MDS space and the users' internal representation of brain anatomy. However, the users quickly learned the general mapping of the 2D MDS space onto the 3D anatomical space such that anterior regions were represented by dots in the upper part of the array whereas dots corresponding to posterior fibers were at the bottom of the display; dots near the midline of the 3D model were at the midline of the dot array and left and right were mapped intuitively.

With this insight, the utility of the 2D MDS panel emerged rather quickly. Large regions containing multiple fibers can be quickly selected for inclusion or removal. For example, removal of an entire hemisphere of fibers is particularly useful when selecting an association tract unilaterally (e.g., superior longitudinal fasciculus). The histogram controls (e.g., length) were useful for reducing clutter initially or after first approximating a selection. The MDS can be monitored while setting thresholds with histogram tools to avoid removal of fibers that might be obscured in the 3D space. In certain situations, it was easier to detect and select errant fibers in the MDS than in the 3D display. The MDS is particularly useful for identifying fibers that were captured by the brush that run in a direction that is orthogonal to that of the desired fibers or that may be hidden from view in the 3D space because they lie in the opposite hemisphere and are thus obscured by other fibers. Fibers selected on the basis of the MDS can be refined using the 3D manipulation tools to obtain an optimal representation of a desired track. Fibers that are grossly inaccurate typically appear as "outlier" dots on the MDS layout. These outliers can be easily and quickly selected in the MDS and removed from the model.

The MDS layout was less helpful when fine tuning a tract selection by removing or adding fibers that are similar in terms of their trajectory and curvature. In cases of similar curvature and trajectory, dots in the MDS panel appear very closely placed and are thus difficult to differentiate. In these situations, fine tuning is best done by judicious use of the brush and single fiber tools. The reader is referred to the accompanying video of the dual interface for additional understanding of how the various tools work.

Quantitative tests of intra- and inter-rater reliability were not conducted as part of this test. However, visual inspection of the selections indicated a high degree of agreement within and between raters.

The users found that many of the interactive tools were similar to those found in other software packages such as Brown University's in-house DTI software (BrainApp) [7], [27] (e.g., box-based selection), CINCH [1], [23] (e.g., brush-like tool), and MedINRIA (http://wwwsop.inria.fr/asclepios/) (e.g., box tools, FA thresholding). The 2D MDS display was clearly new. The users found the combination of many tools in a single package to be very convenient, and this added to the overall utility of the new interface. Surely, some tools or features that are part of the other software packages but not incorporated in the dual interface may be useful.

A more rigorous test of the interface would have involved having multiple raters select a pre-specified set of fibers, using multiple software platforms and multiple datasets. Ideally, the raters would be unfamiliar with the packages used in such a study to control for familiarity bias, and time to practice would be standardized across platforms. Moreover, the raters would be blind to which software was being evaluated. Dependent measures would be time to select each fiber bundle, accuracy, and confidence and ease-of-use ratings. In this preliminary user test, direct comparisons with other software packages were not performed as such a test would have necessarily been vulnerable to experimenter bias given the users were not blind to the "hypothesis" that the interface has advantages over other interfaces and due to their prior experience with multiple platforms.

Prior work in our lab, however, provides useful comparisons across several software platforms [29]. In this prior study, four experienced raters (including one in this user study) with good knowledge of white matter anatomy were asked to identify the corpus callosum (in its entirety), the cingulum bundle (unilateral), the superior longitudinal fasciculus (unilateral), and the uncinate fasciculus (unilateral) using BrainApp, CINCH, and MedINRIA. Outcome measures were time to complete the selection and raters' confidence in the accuracy of their selections. The raters were given up to 30 minutes to familiarize themselves with the programs if needed. Average time to complete tract selection ranged from 3 minutes (superior longitudinal fasciculus using CINCH) to 6.25 minutes (cingulum bundle using BrainApp). For each tract, selection times were faster using CINCH than BrainApp and with selection times for MedINRIA being intermediate between the two. Confidence ratings were generally high.

In contrast, time to select these same bundles in our interface was not more than 3 minutes per fiber bundle for each rater with similarly high confidence in the accuracy of selection. The raters clearly preferred this tool to BrainApp, and the one rater who participated in the user study described above preferred this tool to all three of the tools used in that study, although the preference was not as great in comparison to CINCH.

In short, the users found that the software was quite useful for fiber selection and that time and ease in selecting all target tracts was greatly reduced and the results were superior in terms of precision-time tradeoffs when compared to other platforms with which they were familiar. The users agreed that although the 2D MDS display was initially not recognized to be of additional benefit, its utility was quickly appreciated. The 2D MDS display clearly provides a useful complement to the other tract selection tools. In certain situations, it provides a rapid method for identifying and removing unwanted fibers and a useful means of ensuring that wanted fibers are not inadvertently removed. The interface also appeared to hold promise for improving reliability in DTI fiber tract selection. That is, both raters were able to replicate their own and their counterparts' selections quickly and consistently.

SECTION 6

## Conclusion

We have presented a novel interface that allows for effective representation and intuitive exploration of fiber tracts. At the core of our work is the embedding of the DTI fiber tracts to a 2D space that preserves the characteristics of the fiber tracts. The users can interact with DTI fibers in this 2D space and view, validate, and compare the results of the interaction in both views. Several case studies have verified the efficiency and reliability of the proposed approach.

### Acknowledgments

This work is partially supported by 973 program of China (2009CB320800), NSF of China (No.60873123), the Research Initiation Program at Mississippi State University. Stephen Correia's effort on this project was partly supported by the United States Department of Veterans Affairs. The contents of this manuscript do not represent the views of the Department of Veterans Affairs or the United States.

## Footnotes

Song Zhang is with the Department of Computer Science and Engineering, Mississippi State University, E-mail: szhang@cse.msstate.edu.

Anna MacKay-Brandt and Stephen Correia are with Brown University, E-mail: annamackay@brown.edu, stephencorreia@brown.edu.

Huamin Qu is with the Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, E-mail: huamin@cse.ust.hk.

John Allen Crow is with the College of Veterinary Medicine, Mississippi State University, E-mail: crow@cvm.msstate.edu.

David F. Tate is with Brigham and Women's Hospital, E-mail: dftatephd@mac.com.

The first three authors contribute equally to this work.

Manuscript received 31 March 2009; accepted 27 July 2009; posted online 11 October 2009; mailed on 5 October 2009.

## References

1. CINCH: a cooperatively designed marking interface for 3D pathway selection.

D. Akers

In Proceedings of ACM UIST, pages 33–42, 2006.

2. In vivo fiber tractography using DT-MRI data.

P. J. Basser, S. Pajevic, C. Pierpaoli, J. Duda and A. Aldroubi

Magnetic Resonance in Medicine, 44: 625–632, 2000.

3. Fast and reproducible fiber bundle selection in DTI visualization.

J. Blaas, C. Botha, B. Peters, F. Vos and F. Post

In Proceedings of IEEE Visualization, pages 59–64, November 2005.

4. Modern Multidimensional Scaling: Theory and Applications

I. Borg and P. Groenen

Springer, 2003.

5. Clustering fiber tracts using normalized cuts.

A. Brun, H. Knutsson, H. J. Park, M. E. Shenton and C.-F. Westin

In Proceedings of MICCAI, pages 368–375, September 2004.

6. Abstractive representation and exploration of hierarchically clustered diffusion tensor fiber tracts.

W. Chen, S. Zhang, S. Correia and D. S. Ebert

Computer Graphics Forum, 27 (3): 1071–1078, 2008.

7. Quantitative tractography metrics of white matter integrity in diffusion-tensor MRI.

S. Correia, S. Y. Lee, T. Voorn, D. F. Tate, R. H. Paul, S. Zhang, S. P. Sallowaya, P. F. Malloy and D. H. Laidlaw

Neuroimage, 42: 568–581, 2008.

8. Similarity coloring of DTI fiber tracts.

C. Demiralp and D. H. Laidlaw

In Proceedings of DMFC Workshop at MICCAI, 2009.

9. Connectivity-aware sectional visualization of 3D DTI volumes using perceptual flat-torus coloring and edge rendering.

C. Demiralp, S. Zhang, D. Tate, S. Correia and D. H. Laidlaw

In Proceedings of Euro-graphics, 2006.

10. Visualization of white matter tracts with wrapped streamlines.

F. Enders, N. Sauber, D. Merhof, P. Hastreiter, C. Nimsky and M. Stamminger

In Proceedings of IEEE Visualization, pages 51–58, 2005-10.

11. Uncertainty in white matter fiber tractography.

O. Friman and C.-F. Westin

In Proceedings of MICCAI, pages 107–114, September 2005.

12. The Elements of Statistical Learning

T. Hastie, R. Tibshirani and J. H. Friedman

Springer, 2003.

13. Visualizing spatial relations between 3D-DTI integral curves using texture patterns.

D. Jianu, W. Zhou, C. Demiralp and D. H. Laidlaw

In Proceedings of IEEE Visualization Poster Compendium, 2007.

14. Exploring 3D DTI Fiber Tracts with Linked 2D Representations.

R. Jianu, C. Demiralp and D. H. Laidlaw

IEEE Transactions on Visualization and Computer Graphics, 15 (5), In Press, 2009.

15. Visualization and analysis of diffusion tensor fields.

G. Kindlmann

PhD Dissertation, School of Computing, University of Utah, 2004.

16. Automated atlas-based clustering of white matter fiber tracts from DTMRI.

M. Maddah, A. Mewes, S. Haker, W. Grimson and S. Warfield

In Proceedings of MICCAI, pages 188–195, 2005.

17. Illuminated lines revisited.

O. Mallo, R. Peikert, C. Sigg and F. Sadlo

In Proceedings of IEEE Visualization, pages 19–26, 2005-10.

18. Evaluation of fiber clustering methods for diffusion tensor imaging.

B. Moberts, A. Vilanova and J. J. van Wijk

In Proceedings of IEEE Visualization, November 2005.

19. NVidia

CUDA BLAS Library. http://www.nvidia.com, 2009.

20. Automatic tractography segmentation using a high-dimensional white matter atlas.

L. O'Donnell and C.-F. Westin

IEEE Transactions on Medical Imaging, 26 (11): 1562–1575, 2007.

21. Reconstruction and visualization of fiber and laminar structure in the normal human heart from ex vivo diffusion tensor magnetic resonance imaging (dtmri) data.

D. Rohmer, A. Sitek and G. Gullberg

Invest Radiol., 42 (11): 777–789, 2007.

22. Topological visualization of brain diffusion MRI data.

T. Schultz, H. Theisel and H.-P. Seidel

IEEE Transactions on Visualization and Computer Graphics, 13 (6): 1496–1503, 2007.

23. Exploring connectivity of the brain's white matter with dynamic queries.

A. Sherbondy, D. Akers, R. Mackenzie, R. Dougherty and B. Wandell

IEEE Transactions on Visualization and Computer Graphics, 11 (4): 419– 430, 2005.

24. Fiber tract clustering on manifolds with dual rooted-graphs.

A. Tsai, C.-F. Westin, A. Hero and A. Willsky

In Proceedings of IEEE Computer Vision and Pattern Recognition, pages 1–6, 2007-06.

25. A novel interface for higher-dimensional classification of volume data.

F.-Y. Tzeng, E. B. Lum and K.-L. Ma

In Proceedings of IEEE Visualization, pages 66–74, 2003.

26. Identifying white-matter fiber-bundles in DTI data using an automated proximity-based fiber-clustering method.

S. Zhang, S. Correia and D. H. Laidlaw

IEEE Transactions on Visualization and Computer Graphics, 14 (5): 1044–1053, 2008.

27. Visualizing diffusion tensor MR images using streamtubes and streamsurfaces.

S. Zhang, C. Demiralp and D. H. Laidlaw

IEEE Transactions on Visualization and Computer Graphics, 9 (4): 454–462, 2003.

28. HyperLIC.

X. Zheng and A. Pang

In

Proceedings of IEEE Visualization, pages 249–256, 2003.

29. Evaluation and design for interactive 3d tracts-of-interest selection techniques in diffusion imaging datasets.

W. Zhou

Master Thesis, Brown University, 2008.

30. Semi-supervised learning using gaussian fields and harmonic functions.

X. Zhu, Z. Ghahramani and J. Lafferty

In Proceedings of ICML, pages 912–919, 2003.

## Cited By

No Citations Available

## Keywords

### IEEE Keywords

No Keywords Available

### More Keywords

No Keywords Available

No Corrections

## Media

Video

6,883 KB
Video

6,359 KB
Video

2,552 KB
Video

1,053 KB
Video

7,372 KB
Video

2,955 KB
Video

2,538 KB
Video

### volfilter

3,285 KB
This paper appears in:
IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS
Issue Date:
November/December 2009
On page(s):
1433 - 1440
ISBN:
1077-2626
Print ISBN:
N/A
INSPEC Accession Number:
10930751
Digital Object Identifier:
10.1109/TVCG.2009.112
Date of Current Version:
01 Nov, 2009
Date of Original Publication:
23 Sep, 2009

Rampone, S.