An Iterative Mixed Pixel Classification for Brain Tissues and White Matter Hyperintensity in Magnetic Resonance Imaging

White matter hyperintensities (WMH) generally can be detected and diagnosed by magnetic resonance imaging (MRI). It has been pointed out that WMH is closely associated with stroke, cognitive impairment, dementia, and even is very relevant to the increased risk of death. This paper proposes a new iterative linearly constrained minimum variance (ILCMV) classification-based method which expands an iterative constrained energy minimization (ICEM) detection-based method developed for hyperspectral image classification. It explores the potential of ILCMV combined with different spatial filters in classification of brain normal tissues and WMH and also develops an alternative version of ILCMV, called Multi-class ICEM (MCICEM) for a comparative study. The synthetic images in BrainWeb are used for quantitative evaluation of ILCMV and the real brain MR images are used for visual assessment. The experimental results suggest that the Gaussian filter is most suitable for ILCMV and MCICEM if the computational time is factored into consideration. Otherwise, ILCMV/MCICEM combined with a Gabor filter yields the best classification. In addition, the average Dice similarity indexes (DSI) of CSF/GM/WM volume measurement produced by ILCMV method combined with Gaussian filter were 0.936/0.948/0.975 in synthetic MR images with all noise levels and were better than the results reported in the literature. ILCMV can simultaneously classifies brain normal tissues and WMH lesions in MR brain images and does better than detection of WMH alone. In addition, its computational time is also less than MCICEM. It is our belief that the proposed methodology demonstrates its promising in classification of brain tissue and WMH in MRI applications.

SECTION I. Introduction

The detection of white matter hyperintensity (WMH) by magnetic resonance imaging (MRI) can be used to diagnose stroke, cognitive impairment, dementia, and even the risk of death as pointed out in the past literature [1], [2]. In general, WMH will be present in brighter areas on the T2-weighted or fluid attenuated inversion recovery (FLAIR) MR images. Clinicians make a diagnosis through visual inspection and quantify WMH by manual selection. If a larger volume of WMH appear in the patient’s brain, it will affect the patient’s walking and its motor function [3]. In addition, some works reported in the literature pointed out that the total volume of WMH in subcortical area was related to the rate of decline in cognitive and memory [4]. Therefore, if the volume of WMH can be accurately detected and quantified, it will have substantial significance in assisting clinical diagnosis. Since it is too cumbersome and also not feasible in practical diagnosis to depict the WMH area manually, developing computer aided tools can reduce the complexity of the diagnosis and help clinicians have better diagnosis and monitoring.

In recent years, multispectral image processing technology has been applied to magnetic resonance (MR) image analysis [5]–​[12]. It is different from the spatial domain-based image processing because multispectral MR images containing rich spatial and spectral information from different pulse sequences. The advantage of this technology is that the same substance has different feature attributes to different pulse sequence images that can be used to distinguish different material substances. So, it has been shown that multispectral image processing technology is more efficient than the traditional spatial-domain technology [13]–​[15]. Despite that multispectral image processing technology has its unique advantages, it only takes into the spectral information of the substance from different pulse sequences images and does not consider the spatial information among the substances. Therefore, Chen et al. [16] proposed a hyperspectral image processing technique, called iterative constrain energy minimization (ICEM), which combines the spectral and spatial information of substances to detect WMH. It used a Gaussian filter to capture the spatial information around the detected WMH, and then iteratively feeds the Gaussian filtered spatial information back to the original hyperspectral cube data, until it finally meets a stopping rule to terminate the iterative process.

However, there are still three important issues in Chen et al. [16] that were not discussed. First, since the ICEM method is a detection technique, it can only detect one substance at a time. Therefore, in our case the ICEM method needs to be performed for each of brain substances. A second issue arises from pixel classification problems because ICEM is designed as a detector rather than a classifier. As a result, the literature [16] can only use ICEM to detect WMH substances. If we use ICEM to classify the four substances in the brain, such as gray matter (GM), white matter (WM), cerebrospinal fluid (CSF) and WMH, we must erform ICEM four times to obtain four abundance maps of the substances. Furthermore, since every pixel in these abundance maps is a real value (we call these maps as soft decisions), the issue of how to convert such soft decisions to hard decisions for classification, referred to as mixed pixel classification, becomes a challenge. The final issue is that in addition to Gaussian filter used to acquire WMH spatial information [16] there are also other filters that can be used to capture spatial information but were not discussed in [16].

In this paper, we propose a novel “iterative linearly constrained minimum variance (ILCMV)” method to address the above issues. Linearly constrained minimum variance (LCMV) has been widely used technique to detect multi-component composition in remote sensing imagery [17]–​[19]. ILCMV is developed to improve the detection performance of LCMV through capturing spatial information. Its idea is similar to that used to develop ICEM in [16] to use a set of spatial filters to capture the spatial contextual information and further feed back the spatial filtered maps as new band images to be added to the current image cube to create a new image cube for next round processing of LCMV. The same procedure is repeatedly by feedbacks in an iterative manner. To terminate ILCMV an automatic stopping rule is also described in section 2.5.

The advantage of ILCMV is to simultaneously obtain abundance fractional maps of four substances including normal tissues and WMH in the brain. The maximum abundance value of each pixel decides its class, referred to as maximum a posteriori (MAP). In this paper various different filters, such as Gaussian, Gabor, Guided, and Bilateral filters are used to obtain spatial information around the spectral-classified maps of brain substances. Finally, experiments are conducted to demonstrate that ILCMV using various spatial filters can be used as a quantitative evaluation tool to calculate the abundance fractional maps of GM, WM, CSF and WMH via the synthetic MR image data from BrainWeb (http://brainweb.bic.mni.mcgill.ca/brainweb/) and real MR image data from Taichung Veterans General Hospital.

Three main contributions of this paper can be summarized as follows. First of all, our proposed method, ILCMV, can simultaneously detect and classify brain substances, and thus reduce the operation time of ICEM in [16]. Second, we use MAP to determine the class of each pixel. In other words, the ILCMV calculated abundance fractions can provide valuable information as soft decisions that can enhance visual contrast of physicians. In addition, such ILCMV generated abundance fractional maps can be further used to perform pixel classification as hard decisions via a thresholding technique such as Otsu’s method [20]. This concept has not been explored in the past. The third contribution is to gain insight into the influence of ILCMV using different spatial filters on the confidence level of test data.

SECTION II. Materials and Methods

A. Materials-Brain Synthetic MR Image and Real MR Image Data

The synthetic MR images with multiple sclerosis (MS) were downloaded from BrainWeb, which was developed by McConnell Brain Imaging Centre, McGill University, Canada. Figure 1(a) is one slice of the synthetic MR images with MS lesions, and Figure 1(b) shows the ground truths of brain tissues and MS lesions. MS lesions are typically hyperintense on T2 weighted (T2W) or fluid attenuation inversion recovery (FLAIR) sequence image. These brain MR images are included in the modalities of proton density (PD), T1 weighted (T1W), and T2 weighted (T2W) with specifications provided in BrainWeb. The thickness of slice is 1 mm with size of $181\times 217\times181$ . Each slice is specified by INU (intensity non-uniformity) 0% or 20%, denoted by rf0 and rf20 with six different levels of noise, 0%, 1%, 3%, 5%, 7% and 9%.

FIGURE 1.

The synthetic brain MR images with Multiple Sclerosis (MS) from BrainWeb. (a) From left to right is the PD, T1W, and T2W MR image respectively with 0% noise level and 0% INU. (b) The ground truths of CSF, GM, WM, and MS lesions.

Our study also used real brain MR images with different grades of lesion distributions for analysis. The distributions of lesions were divided into small, medium and large levels by Fazekas et al. [21]. Figure 2 showed the different Fazekas levels of white matter hyperintensity (WMH) in FLAIR images. In this study, the 1.5 T whole body magnetic resonance machine manufactured by Siemens (Germany) was used to obtain the following three sets of three-dimensional brain MR images. The details of the imaging parameters are as follows: T1-weighted (T1w) 3D MRI (MP-RAGE): TR/TE = 1600ms/3–5ms, 1 excitation; T2-weighted (T2w) turbo-spin-echo (TSE) SPACE: TR/TE = 3200–4000ms/360–400ms, 1 excitation, with variable flip-angle distribution; FLAIR 3D MRI: TR/TI = 5000ms / 1800ms, TE = 357ms, one excitation, with variable flip-angle distribution. All of them have the same parameters setting such as slice thickness = 1.1mm, matrix $224\times256$ , field-of-view = 22–24cm. In addition, the research conducted in this paper was approved by the Clinical Research Ethics Committee of Taichung Veterans General Hospital (IRB No.: CE16138A). Of the 111 cases collected, 58 cases belonged to Fazekas level 1, 44 cases belonged to Fazekas level 2, and the rest were Fazekas level 3. Therefore, we selected 10 cases from Fazekas level 1, 11 cases from Fazekas level 2, and 9 cases from Fazekas level 3 for analysis and comparison.

FIGURE 2.

Lesion categorized by three grades of Fazekas shown in FLAIR images. (a) Fazekas grade 1. (b) Fazekas grade 2. (c) Fazekas grade 3.

B. Preprocessing - Nonlinear Band Expansion (NBE)

Since ICEM and ILCMV are hyperspectral imaging techniques which require a large number of band images and the used brain MR images have only three band images acquired by T1-weigted, T2-weighted and FLAIR, there are not sufficient images that can be used for ICEM or ILCMV. To address this issue a nonlinear band extension (NBE) must be implemented prior to ICEM/ILCMV. The NBE can expand the original MR image to more band images through nonlinear functions such as auto- or cross-correlation. Combining these new NBE-generated images with the original image produces a new set of hyperspectral images that can be used by ICEM and ILCMV. In this paper, we use the $3^{\mathrm {rd}}$ order correlation band images as the pre-processing step [16] for NBE. It is worth noting that NBE can only generate nonlinear spectral information of brain tissues or WMH, but cannot capture their spatial information. Therefore, we will use different spatial filters to extract spatial information to be fed back to ILCMV. The details of the implementation are described in the following section.

C. Linearly Constrained Minimum Variance (LCMV) [18]

We interpret the six MR images, T1-weighted, T2-weighted, FLAIR, and three $3^{\mathrm {rd}}$ order correlation-generated band images from NBE as spectral bands, in which case MR images can be then considered as a hyperspectral image cube. Let $\left \{{{\mathbf {r}_{nL}} }\right \}_{n=1}^{N}$ be an $L$ -dimensional MR image pixel vector where $L$ is the number of image pulse sequences used for MR data acquisition and each image acquire by a particular pulse sequence or the NBE preprocessing is considered as a spectral band image. In addition, $N$ is the number of all pixels in MR image, and the $n^{th}$ pixel vector is $\mathbf {r}_{n} =\left ({{r_{n1},\;r_{n2},\;\ldots,r_{nL}} }\right)^{T}for\;1\le n\le N$ . We assume that gray matter (GM), white matter (WM), cerebrospinal fluid (CSF), and WMH are interest targets and their spectral signature vectors are $\left \{{{{\mathbf { d}}_{j},\;j=1,\,2,\,3,\,4} }\right \}$ . Therefore, we can obtain an interest target signature matrix specified by ${\mathbf { D}}=\left [{ {{\mathbf { d}}_{1},{\mathbf { d}}_{2},{\mathbf { d}}_{3},{\mathbf { d}}_{4}} }\right]$ . Our goal is to design a linear finite impulse response (FIR) filter which is a $L$ -dimension weighted vector specified by $\mathbf {w}=\left ({{w_{1},w_{2},\ldots w_{L}} }\right)^{T}$ to minimize the filter output energy subject to the equation (1).

$$\begin{equation*} \mathbf {D}^{T}\mathbf {w}=\mathbf {c} ~\text {where}~ {\mathbf { d}}_{j}^{T} \mathbf {w}=\sum \limits _{l=1}^{L} {w_{l} d_{jl}} =c_{j} \quad \mathbf {for}1\le j\le 4\tag{1}\end{equation*}$$ where $\mathbf {c}=\left ({{c_{1},c_{2},c_{3},c_{4}} }\right)^{T}$ is a constraint vector, ${l}$ is one of spectral bands and $L$ is the number of all spectral bands. Let $y_{n}$ is the output signal of each input pixel vector, $\mathbf {r}_{n}$ , through the designed FIR filter, and $y_{n}$ is shown as the equation (2). $$\begin{equation*} y_{n} =\sum \limits _{l=1}^{L} {w_{l} r_{nl}} =\mathbf {w}^{T}\mathbf {r}_{n} =\mathbf {r}_{n}^{T} \mathbf {w}\tag{2}\end{equation*}$$

Based on the concept of beam-former from LCMV, we can find the average energy of the output signal as shown in the equation (3).

$$\begin{align*} \frac {1}{N}\sum \limits _{n=1}^{N} {y_{n}^{2}}=&\frac {1}{N}\sum \limits _{n=1}^{N} {y_{n}^{T} y_{n}} =\frac {1}{N}\sum \limits _{n=1}^{N} {\left ({{\mathbf {r}_{n}^{T} \mathbf {w}} }\right)^{T}\left ({{\mathbf {r}_{n}^{T} \mathbf {w}} }\right)} \\=&\mathbf {w}^{T}\left ({{\frac {1}{N}\sum \limits _{n=1}^{N} {\mathbf {r}_{n} \mathbf {r}_{n}^{T}}} }\right)\mathbf {w}=\mathbf {w}^{T}\mathbf {R}_{L\times L} \mathbf {w}\tag{3}\end{align*}$$

Therefore, we design an LCMV-based desired target detector that can be constrained by the minimum of equation (1), as shown in equation (4).

$$\begin{equation*} \min \limits _{\mathbf {w}} \left \{{{\mathbf {w}^{T}\mathbf {R}_{L\times L} \mathbf {w}} }\right \} ~\text {subject to}~ \mathbf {D}^{T}\mathbf {w}=\mathbf {c}\tag{4}\end{equation*}$$ where $\mathbf {R}_{L\times L}$ is an autocorrelation sample matrix in hyperspectral images. The equation (4) can be considered as a constrained least mean squares problem, so we can use the Lagrange multiplier method to fine the optimal solution weight $\mathbf {w}^{opt}$ . First, the constraint function and cost function from equation (3) combined to a $H\left ({{\mathbf {w}} }\right)$ function through a Lagrange multiplier vector ${\boldsymbol \lambda }$ . For the convenience of operations, we put the constant $\frac {1}{2}$ in the cost function, so it can be rewritten as equation (5). $$\begin{equation*} H\left ({{\mathbf {w}} }\right)=\frac {1}{2}\mathbf {w}^{T}\mathbf {R}_{L\times L} \mathbf {w}+\boldsymbol {\lambda } ^{\mathbf {T}}\left ({{\mathbf {D}^{T}\mathbf {w}-\mathbf {c}} }\right)\tag{5}\end{equation*}$$

We take the gradient of equation (5) for w, and then we can get the equation (6).

$$\begin{equation*} \nabla _{\mathbf {w}} H\left ({{\mathbf {w}} }\right)=\mathbf {R}_{L\times L} \mathbf {w}+\boldsymbol {\lambda } ^{\mathbf {T}}\mathbf {D}^{T}=\mathbf {R}_{L\times L} \mathbf {w}+\mathbf {D}\boldsymbol {\lambda }\tag{6}\end{equation*}$$

Therefore equation (6) is equal to zero if the optimization is required, that is $\nabla _{\mathbf {w}} H\left ({{\mathbf {w}} }\right)=\mathbf {R}_{L\times L} \mathbf {w}+\mathbf {D}\boldsymbol {\lambda }= 0$ . For the Lagrange multiplier, the optimal weight vector is shown in equation (7).

$$\begin{equation*} \mathbf {w}^{opt} =\mathbf {-R}_{L\times L}^{-1}\mathbf {D}\boldsymbol {\lambda }\tag{7}\end{equation*}$$

Since we assume $\mathbf {R}_{L\times L}$ is a positive definite matrix, $\mathbf {R}_{L\times L} ^{-1}$ must be existed, and $\mathbf {w}^{opt}$ is satisfied the condition of equation (4). So $\mathbf {D}^{T}\mathbf {w}^{opt}=\mathbf {D}^{T}\left ({\mathbf {-R}_{L\times L}^{-1}\mathbf {D}\boldsymbol {\lambda } }\right)=\mathbf {c}$ , and then we can find the Lagrange multiplier vector ${\boldsymbol \lambda }$ as shown in equation (8).

$$\begin{equation*} \boldsymbol {\lambda } =-\left [{ {\mathbf {D}^{T}\mathbf {R}_{L\times L} ^{-1}\mathbf {D}} }\right]^{-1}\mathbf {c}\tag{8}\end{equation*}$$

From equations (7) and (8), we can find the optimal weight vector $\mathbf {w}^{opt}$ of constrained least mean squares problem, such as equation (9).

$$\begin{equation*} \mathbf {w}^{opt} =\mathbf {R}_{L\times L}^{-1}\mathbf {D}\left [{ {\mathbf {D}^{T}\mathbf {R}_{L\times L}^{-1}\mathbf {D}} }\right]^{-1}\mathbf {c}\tag{9}\end{equation*}$$

We replace the weighted vector $\mathbf {w}$ in (2) by the optimal weight vector $\mathbf {w}^{opt}$ in the LCMV filter, which implements a detector, $\delta ^{\mathbf {LCMV}}\left ({{\mathbf {r}} }\right)$ given by equation (10).

$$\begin{equation*} \delta ^{\mathbf {LCMV}}\left ({{\mathbf {r}} }\right)=\left ({{\mathbf {w}^{opt}} }\right)^{T}\mathbf {r}\tag{10}\end{equation*}$$

By virtue of (10) the abundance maps of brain tissue and abnormal area such as GM, WM, CSF, and WMH, ca be generated for data analysis.

D. Iterative LCMV (ILCMV)

This section introduces an iterative version of LCMV, referred to as ILCMV. Its main idea is to implement spatial filters on the LCMV-generated abundance maps to capture spatial information around LCMV-classified pixels. Then these spatial filtered LCMV-classified maps are further fed back to be added to the current being processed image cube to create a new set of image cubes for next round LCMV processing. The iterative process will be terminated once a stopping rule is met. Figure 3 depicts a flowchart of step-by-step ILCMV implementation. When the original brain MR images are expanded by NBE to generate a new set of brain MR image cubes, these images are then used as an input to ILCMV to obtain the abundance fractional maps of four components of interest. Once ILCMV satisfies the stopping rule, the resulting LCMV-classified abundance fractional maps are converted by MAP to decide the classes of pixels. The dashed box in Figure 3 describes the stopping rule of how the iterative process is terminated.

FIGURE 3.

A flowchart of ILCMV.

The details of ILCMV implemented in Figure 3 are summarized as follows:

ILCMV

1. Initial conditions: Let $\Omega ^{\left ({0 }\right)}$ is a set of brain hyperspectral MR images after the NBE preprocessing, and $\mathbf {D}^{\left ({0 }\right)}$ is a set of spectral profiles of brain tissue and WMH which we are interested in. We make $k=1$ .

2. After executing $\delta _{k}^{\mathbf {LCMV}}$ with $\mathbf {D}^{\left ({k }\right)}$ on the set $\boldsymbol {\Omega }^{\left ({k }\right)}$ , a set $\mathbf {A}_{k}^{\mathbf {LCMV}}$ of LCMV abundance maps is generated. This set is contained four abundance maps for gray matter, white matter, cerebrospinal fluid, and WMH.

3. Use different spatial filters to smooth the abundance map set $\left |{ {\mathbf {A}_{k}^{\mathbf {LCMV}}} }\right |$ , where $\left |{ {\mathbf {A}_{k}^{\mathbf {LCMV}}} }\right |$ is the absolute value of the abundance map set $\mathbf {A}_{k}^{\mathbf {LCMV}}$ . These smoothing abundance images by spatial filters, such as Gaussian, Gabor, Guided, or Bilateral, are defined as $\vert \mathbf {F}{\mathbf { B}}_{k}^{\mathbf {LCMV}} \vert$ .

4. We can obtain the smoothing abundance image set $\vert \mathbf {F}{\mathbf { B}}_{k}^{\mathbf {LCMV}} \vert$ after the ($k$ -1)$^{\mathrm {th}}$ brain MR hyperspectral image set $\boldsymbol {\Omega }^{\left ({{k-1} }\right)}$ is calculated by the LCMV method. And then we combine this set of images $\vert \mathbf {F}{\mathbf { B}}_{k}^{\mathbf {LCMV}} \vert$ with the ($k$ -1)$^{\mathrm {th}}$ image set $\boldsymbol {\Omega }^{\left ({{k-1} }\right)}$ to obtain the $k^{\mathrm {th}}$ new brain MR hyperspectral image set $\boldsymbol {\Omega }^{\left ({k }\right)}$ . The mathematical expression is defined as $\boldsymbol {\Omega }^{\left ({k }\right)}=\boldsymbol {\Omega }^{\left ({{k-1} }\right)}\cup \left |{ {\mathbf {FB}_{k}^{\mathbf {LCMV}}} }\right |$ .

5. Verify that the abundance map set $\left |{ {\mathbf {A}_{k}^{\mathbf {LCMV}}} }\right |$ meets the stop rules described in Section 2.5. If it does, we will stop the iteration loop and skip to step 7. If not, go back to step 2 and continue with ILCMV.

6. The obtained abundance image set $\vert \mathbf {F}{\mathbf { B}}_{k}^{\mathbf {LCMV}} \vert$ and the previous brain MR hyperspectral image set are combined into a new operation set, such as $\boldsymbol {\Omega }^{\left ({{k+1} }\right)}=\boldsymbol {\Omega }^{\left ({k }\right)}\cup \left |{ {\mathbf {FB}_{k}^{\mathbf {LCMV}}} }\right |$ . Let $k\leftarrow k+1$ and returns to Step 2.

7. End of the iteration of ILCMV, use the winner-take-all condition, i.e., MAP, to classify each pixel of the abundance map into brain tissue or WMH on binary image.

E. Stopping Rules of ILCMV

In order to effectively stop ILCMV, we use Otsu’s method [20] to convert the abundance fractional maps of $\left |{ {\mathbf {A}^{\mathbf {LCMV}}} }\right |$ of GM, WM, CSF and WMH into their corresponding binary maps $\left |{ {\mathbf {B}^{\mathbf {LCMV}}} }\right |$ . We then use the Dice similarity index (DSI) [22] defined as follows as a measure for stopping rule

$$\begin{equation*} \mathbf {DSI}^{\left ({k }\right)}=\,\left ({{\frac {2\left |{ {B_{k} \cap {B}_{k-1}} }\right |}{\left |{ {B_{k} {\vert +\vert B}_{k-1}} }\right |}} }\right)\tag{11}\end{equation*}$$ where $\mathbf {B}_{k}$ and $\mathbf {B}_{k-1}$ represent the binary result set of the $k^{\mathrm {th}}$ and ($k-1)^{\mathrm {th}}$ LCMV methods, respectively. The DSI is the average of four binary results since each abundance fractional map set can produce four binary results. When the averaged DSI is greater than threshold $\varepsilon (\varepsilon$ is a similar threshold, here we set 0.99, 0.95, 0.90, 0.85, and 0.80), the iterative process is terminated and the final abundance fractional maps of ILCMV binarized by MAP. Figure 4 delineates a flowchart of the stopping rule implemented in ILCMV.

FIGURE 4.

A flow chart of a stopping rule.

F. Multiple Class Iterative Constrained Energy Minimization (MCICEM)

The ICEM proposed by Chen et al. [16] was originally designed to detect white matter hyperintensity area while discarding detection of other brain tissues, such as GM, WM, CSF, etc. In order to compare with ILCMV, we expand ICEM into a multi-class ICEM (MCICEM). Despite the fact that MCICEM and ILCMV are both multi-class classification methods, a key difference between them that MCICEM can only feed back abundance fractional map of one target at a time in ICEM, while ILCMV can feed back abundance fractional maps of all targets all together in a feedback loop. In other words, MCICEM is extension to ICEM as a multiple-target detector. In this case, the feedback loops are carried out by the number of targets via constraining each desired target signature d_j in (1). By contrast, ILCMV is a multiple-class classifier which classifies all targets simultaneously in one shot operation via the desired matrix D in (9). As a result, there is only one single feedback loop resulting from ILCMV.

There are two scenarios of implementing a stopping rule in MCICEM. One is that the DSI value of each target needs to meet the same threshold value $\varepsilon$ for all targets (where we set the same threshold as 0.99, 0.95, 0.90, 0.85 and 0.80).The resulting MCICEM is referred to as MCICEM. The other is that the DSI value of each target varies. In this case, the process of each target will be terminated at a different number of iterations. The process will be only terminated until all targets are completed. Such MCICEM is referred to as MCICEM-4DSI. Since MCICEM-generated abundance fractional maps are also soft decisions, we need to use MAP to make hard decisions for each pixel in the same way as ILCMV does. Figure 5 provides a flow chart of MCICEM. More details of ICEM are referred to [16].

FIGURE 5.

An implemented flow chart of MCICEM.

G. Filters Used to Capture Spatial Information of Brain Tissues and WMH

In this section, four different spatial filters, Gaussian, Gabor, Guided and Bilateral filters are used to explore their effects on capturing spatial information of brain tissues and WMH.

1) Gaussian Filter

The Gaussian filter is the most commonly used smoothing spatial filter in digital image processing [23], which is mainly used to blur the edges of images to reduce the sharp transition of grayscale intensities in images. Its formula is shown in equation (12).

$$\begin{equation*} G\left ({{x,y} }\right)=\frac {1}{2\pi \sigma ^{2}}e^{-\frac {x^{2}+y^{2}}{2\sigma ^{2}}}\tag{12}\end{equation*}$$

2) Gabor Filter

In the field of image processing, many scientists believe that the frequency and direction of the Gabor filter can be used to simulate the human visual system. It is found that the Gabor filter is particularly suitable for texture representation and discrimination. The 2D-Gabor filter can simulate human two-dimensional visual perception by the tuning characteristics of spatial positioning, direction selectivity, spatial frequency selection and quadrature phase relationship. This neural model was originally proposed by Daugman [24], [25]. Its formula is shown in equation (13).

$$\begin{align*}&\hspace {-2pc}f\left ({{x,y} }\right)=\exp \left \{{{-\pi [\left ({{x-x_{0}} }\right)^{2}a^{2}+\left ({{y-y_{0}} }\right)^{2}b^{2}]} }\right \} \\&\qquad \times \exp \left \{{{-2\pi i[u_{0} \left ({{x-x_{0}} }\right)+v_{0} \left ({{y-y_{0}} }\right)] } }\right \}\tag{13}\end{align*}$$

The spatial position parameters in equation (13) are ($x_{0},y_{0}$ ), the modulation parameters are ($u_{0},v_{0}$ ) and two scale parameters are ($a,b$ ).

3) Guided Filter

The guided filter is also one of commonly used smoothing filters. The output of a guided filter is a linear transform by using the guidance image, and it has better performance and the fastest edge-preserving properties [26]. Therefore, this paper also uses it as one of capturing spatial information filters, as shown in equation (14).

$$\begin{equation*} q_{i} =\sum \limits _{j} {W_{ij} \left ({G }\right)p_{j}}\tag{14}\end{equation*}$$ where $p$ represents the input image, $q$ represents the output image, $G$ represents the guided image, and $i$ and $j$ are pixel indexes. The filter kernel $W_{ij}$ is a function of the guidance image, and is independent of the input image $p$ .

4) Bilateral Filter (BF)

The bilateral filter is a non-linear, edge-preserving and noise-reducing smoothing filter that replaces the intensity value of each pixel, $I\left ({{x_{i},y_{i}} }\right)$ , by the average weighted intensity $w$ from the surrounding pixel $\left ({{x,y} }\right)$ [26], [27]. The weighted function is usually done by a Gaussian function as described in equations (15) and (16).

$$\begin{align*}&\hspace{-2.7pc}BF\left ({{x_{i},y_{i}} }\right)=\frac {1}{w}\sum \nolimits _{\left ({{x,y} }\right)\in w} {G_{\sigma _{s}} \left ({{\left \|{ {\left ({{x,y} }\right)-\left ({{x_{i},y_{i}} }\right)} }\right \|} }\right)} \\&\qquad \qquad \times G_{\sigma _{I}} \left ({{\left |{ {I\left ({{x,y} }\right)-I\left ({{x_{i},y_{i}} }\right)} }\right |} }\right)I\left ({{x_{i},y_{i}} }\right)\tag{15}\end{align*}$$ where $w$ is $$\begin{align*} w\!=\!\!\sum \nolimits _{\left ({{x,y} }\right)\in w} \!\!{G_{\sigma _{s}} \!\left ({{\left \|{ {\left ({{x,y} }\right)\!-\!\left ({{x_{i},y_{i}} \!}\right)} }\right \|} }\right)} G_{\sigma _{I}} \left ({{\left |{ {I\left ({{x,y} }\right)\!-\!I\left ({{x_{i},y_{i}} }\right)} }\right |} }\right)\!\!\!\! \\ {}\tag{16}\end{align*}$$ where $G_{\sigma _{s}}$ and $G_{\sigma _{I}}$ are the one-dimensional Gaussian functions of $G_{\sigma _{s}} \left ({x }\right)=\frac {1}{\sqrt {2\pi } \sigma _{s}}\exp \left ({{-\frac {x^{2}}{2\sigma _{s}^{2}}} }\right)$ and $G_{\sigma _{I} } \left ({x }\right)=\frac {1}{\sqrt {2\pi } \sigma _{I}}\exp \left ({{-\frac {x^{2}}{2\sigma _{I}^{2}}} }\right)$ , respectively.

SECTION III. Results

In this section the performance of ILCMV and MCICEM were compared via the synthetic brain MR images from BrainWeb and real brain MR images from Taichung Veterans General Hospital described in section II.A.

A. Synthetic Brain MR Images

We used four different spatial filters to capture the spatial information of the synthetic brain MR images, and the parameters used in each filter are shown in Table 1. According to the experimental results of Chen et al. [16], the result was better when the window width of the filter is $5\times5$ in higher noises. Therefore, we used the same window width $5\times5$ for all filters as the benchmark comparison. The other parameters of different filters are listed in Table 1. We use the five DSI threshold cases of 0.99, 0.95, 0.90, 0.85 and 0.80 to terminate ICMV and MCICEM. According to our experimental results, the classification performance of using a higher threshold for the stopping rule was better when the image noise was increased. This is true for all the three methods, ILCMV, MCICEM, or MCICEM-4DSI. Due to limited space we only include the results when the threshold value for the stopping rule was set to 0.99. This threshold gave the best results of BrainWeb images in twelve noise levels.

TABLE 1 Parameters Used by ILCMV, MCICEM and MCICEM-4DSI

Both ILCMV and MCICEM used all slices-selected training samples from Chen et al. [16] of CSF, GM, WM and WMH as a priori information to apply to all experiments. These training samples were randomly selected three pixels of each substance by the physician, and then we used spectral angle mapper (SAM) [18] to find the similar pixel vector as desired target vector. The training samples of synthetic image and real image were selected in the same way.

Table 2 tabulates the average DSI values of ILCMV for classifying brain tissues and WMH using four different spatial filters at 12 different levels of noise and non-uniformity. Tables 3 and 4 tabulate the results of MCICEM and MCICEM-4DSI, respectively. The best results are boldfaced in Tables 2 to 4. Table 5 tabulates the iteration numbers of three proposed methods with different spatial filters.

TABLE 2 The Averaged DSI Results of GM, WM, CSF and WMH Quantification in the Brain Synthetic MRI ( $1\times1\times1$ mm $^{3}$ ) at Various Noise and Intensity Uniformity Settings by Using ILCMV With Different Spatial Filters

TABLE 3 The Averaged SI Results of GM, WM, CSF and WMH Quantification in the Brain Synthetic MRI ( $1\times1\times1$ mm $^{3}$ ) at Various Noise and Intensity Uniformity Settings by Using MCICEM With Different Spatial Filters

TABLE 4 The Averaged SI Results of GM, WM, CSF and WMH Quantification in the Brain Synthetic MRI ( $1\times1\times1$ mm $^{3}$ ) at Various Noise and Intensity Uniformity Settings by Using MCICEM-4DSI With Different Spatial Filters

TABLE 5 The Iteration Numbers of Three Proposed Methods With Different Spatial Filters

From the results of Table 2 to Table 4, we found that whether the results of ILCMV and MCICEM with the Gabor filter were always better than their counterparts using other filters except the case that the results using Gaussian filers were better where the noise and non-uniformity are 0%. Two observations can be made from the results of Table 5. First, the numbers of iterations using ILCMV method was smaller than those of the other two methods. Second, as the image noise level was increased, the iterative number of each method was also increased. The Kruskal-Wallis test using multiple sets of sample variance analysis showed that ILCMV, MCICEM and MCICEM-4DSI had no significant differences in classification of CSF, GM, WM and WMH as shown in Table 6. Table 7 also tabulates results of using the Kruskal-Wallis test for ILCMV, MCICEM and MCICEM-4DSI combined with different spatial filters in classification of CSF, GM, WM and WMH. We could see that three methods combined with different spatial filters had significant differences in classification of CSF and WMH. There is only significant difference in classification of GM and WM using the MCICEM-4DSI using different spatial filters. Figure 6 shows the box plots of three methods with different spatial filters that have significant differences in brain normal tissue and WMH classification.

TABLE 6 The Kruskal-Wallis Test Results of ILCMV, MCICEM and MCICEM-4DSI Methods for the Classification of CSF, GM, WM and WMH

TABLE 7 The Kruskal-Wallis Test Results of Gaussian, Gabor, Guided and Bilateral filters for the Classification of CSF, GM, WM and WMH

FIGURE 6.

Box plots of three different methods, combined with four spatial filters from the significant differences, with the results in Table 7.

B. Computational Burden

It is very important for reasonable computing time and hardware requirements in developing algorithms for accurate classification and clinical applications. We used a Windows 7 computer with CPU Intel® Xeon® E5-2620 v3 @ 2.40 GHz processor and 32 GB RAM to evaluate the performance of the proposed methods. The average computational time of different methods combined with four spatial filters for all synthetic MRI data sets is shown in Figure 7. It can be seen that ILCMV has significantly less computational time than the other two methods. As for the different spatial filters, the computing time required by the Gaussian filter was least.

FIGURE 7.

Average computational time of three different methods, combined with four spatial filters in all synthetic MRI data sets.

C. Real Brain MR Images

We also used real brain MRI data from Taichung Veterans General Hospital for experiments to evaluate the performance of our purposed methods, ILCMV, MCICEM and MCICEM-4DSI. The same parameters in Table 1 were also used and the DSI parameter for the stopping rule was set to 0.99.

Fig.s 8 to 10 show the classification results of CSF, GM, WM, and WMHs by using three methods combined with four spatial filters for real brain MR images of three Fazekas grades. From the classification results of these three Fazekas grades, several interesting findings are observed.

FIGURE 8.

The classification results of CSF, GM, WM, and WMH by using ILCMV, MCICEM, and MCICEM-4DSI in real brain MRI with the WMH grade of Fazekas 1. They simultaneously have soft and hard classification results. From left to right in the image are the soft classification result of CSF, GM, WM, and WMH, with the color image being the hard classification result. (A-D) The results of ILCMV method combined with Gaussian, Gabor, Guided and Bilateral filters, respectively. (E-H) The results of MCICEM method. (I-L) The results of MCICEM-4DSI method.

FIGURE 9.

The classification results of CSF, GM, WM, and WMH by using ILCMV, MCICEM, and MCICEM-4DSI in real brain MRI with the WMH grade of Fazekas 2. They simultaneously have soft and hard classification results. From left to right in image are the soft classification result of CSF, GM, WM, and WMH, with the color image being the hard classification result. (A-D) The results of ILCMV method combined with Gaussian, Gabor, Guided and Bilateral filters, respectively. (E-H) The results of MCICEM method. (I-L) The results of MCICEM-4DSI method.

FIGURE 10.

The classification results of CSF, GM, WM, and WMH by using ILCMV, MCICEM, and MCICEM-4DSI in real brain MRI with the WMH grade of Fazekas 3. They simultaneously have soft and hard classification results. From left to right in image are the soft classification result of CSF, GM, WM, and WMH, with the color image being the hard classification result. (A-D) The results of ILCMV method combined with Gaussian, Gabor, Guided and Bilateral filters, respectively. (E-H) The results of MCICEM method. (I-L) The results of MCICEM-4DSI method.

First, the three classification methods, ILCMV, MCICEM, and MCICEM-4DSI combined with Gaussian and Gabor filters were better than using other filters where the classification results using Bilateral filters was the worst among the all real image classification results.

Second, from the results in Figures 8 to 10, especially the red arrow marks in Figures 9 and 10, we can see that ILCMV was better than MCICEM and MCICEM-4DSI in simultaneously classifying brain tissue and WMH. Because WMH was correctly classified when using ILCMV with bilateral filters, and the other methods with bilateral filters were not.

Third, if we only compared the WMH classification result of ILCMV with the results obtained in [16], the results of three methods combined with the Gaussian and Gabor filters are better than those in [16].

Fourth, ILCMV performs classification of the four brain components, such as CSF, GM, WM and WMH, which are superior to MCICEM and MCICEM-4DSI, at least half of the computing time in the total 30 real cases.

SECTION IV. Discussion

This paper proposes a new hyperspectral magnetic resonance imaging processing technique, ILCMV, which can simultaneously classify the normal brain tissue and white matter hyperintensity and can also generate their abundance fractional maps for soft decisions as well as their binary classification maps for hard decision, respectively. In clinical applications, soft decisions can provide assistance for clinicians to diagnose, while hard decisions can produce quantitative results that can be used as an objective data of patients for tracking differences before or after treatment.

The experimental results show that when synthetic MR images were used for experiments there was no significant difference among ILCMV and MCICEM and MCICEM-4DSI in classification of the normal brain tissue and WMH. However, the computing time of ILCMV has been shown to be faster than the other two methods. Time saving is very useful for clinical practical applications.

From Tables 2–​4, the classification results of synthetic MR image ILCMV, MCICEM, and MCICEM-4DSI, using Gabor filters were the best. But the computing time of using Gabor filters was slower than other filters as shown in Figure 7. The fastest and slowest time difference can be 4.3 times or more. Although the Gabor filter is considered to be effective in processing image texture information [24], [25], its computing time is a factor that needs to be considered when it is applied in clinical application.

In the Kruskal-Wallis test results of the three classification methods with the different spatial filters, we can see that there was significant difference in classification of CSF and WMH when different spatial filters were used to capture spatial information around pixels. This is because the difference in edge with high signal to noise ratio can also be blurred by the filters and both CSF and WMH are features of high signal intensity in PD or T2 MR images. So the Kruskal-Wallis test results had significant differences. However, there were no significant differences in the Kruskal-Wallis test results in classification of GM and WM because the intensity of GM and WM in PD or T2 MR images are darker than CSF and WMH.

From the box plot results in Figure 6, the classification results of CSF using ILCMV with four different filters were more robust than the results of the other two methods with four different filters. According to the classification results of WMH, ILCMV with the Gaussian and Gabor filters were more stable than the other two methods. In addition, the three methods combined with the Guided filters made the classification results change greatly with the influence of different noise and non-uniformity levels. This is because the Guided filter is affected by its used guided image and the smoothing effect is not so obvious.

The concept of ILCMV is derived from hyperspectral image processing technology. So, when MR images are considered as multispectral band images, each of these MR band images represents information provided by a different pulse sequence, multispectral images suffer from insufficient band images. In order to resolve this issue, we need to use NBE to expand insufficient band images resulting from multispectral images so that hyperspectral image analysis technology can be readily applied. Despite that such band expansion may not have real physical reasoning it does provide a significant advantage to cope with the issue of insufficient band images. However, due to the length of the paper, the third-order correlation band image is used for NBE in the paper for image analysis and the effects of using different nonlinear band expansion methods are not discussed but can be explored in future applications.

SECTION V. Conclusion

This paper develops a novel approach, called ILCMV, which can simultaneously classify brain normal tissues and WMH lesions in MR brain images. Since accurate and simultaneous classification of brain normal tissue and WMH are very important for assessing the cognitive deficits of dementia patients and the brain tissue changes of the elderly, this paper verifies the accuracy and feasibility of the proposed ILCMV via synthetic brain MR image and real brain MR image experiments. In addition, a comparative analysis between the effects and feasibility of ILCMV and MCICEM using different spatial filters is performed. From the experimental results and computational time, it shows that the Gaussian filter is most suitable for ILCMV and MCICEM. Although using Gaussian filters is slightly worse in accuracy than that using Gabor filters, its computing time is far superior to that using Gabor filters in the sense that they both can simultaneously complete the task of classifying brain normal tissue and WMH. Moreover, ILCMV can further provide the abundance fractional map results (soft decision) which can help the clinician for visual assessment and diagnosis as well as produce binary classification map results (hard decision) for quantifying the brain normal tissue and WMH volume. This advantage has never been explored in clinical MRI research literature in the past.

ACKNOWLEDGMENT

The authors thank Mr. Jun-Peng Chen who works within the Biostatistics Task Force of Taichung Veterans General Hospital for his statistical analysis in this paper.