MRI and CT Medical Image Fusion Based on Synchronized-Anisotropic Diffusion Model

Medical image fusion can combine information from multi-modality images and express them through a single image. How to design a fusion method to preserve more information becomes a hot topic. In this paper, we propose a novel multi-modality medical image fusion method based on Synchronized-Anisotropic Diffusion Equation (S-ADE). First, the modiﬁed S-ADE model which is more suitable for Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) images is employed to decompose two source images. We get the base layers and texture layers. Next, the ‘‘Maximum Absolute Value’’ rule is used for base layers fusion. On texture layers, the fusion decision map is calculated by New Sum of Modiﬁed Anisotropic Laplacian (NSMAL) algorithm which is designed using common decomposition coefﬁcients by anisotropic diffusion. Furthermore, the consistency check is constructed on the decision map to mitigate the staircase effect. After that, the fused image is obtained by a simple linear combination of layers. Finally, the fused MR/CT image is obtained after image correction. Its aim is to eliminate redundant texture information which is from MRI images in the contour part. The extensive experimental results demonstrate that the proposed method preserves much information as well as guarantees image quality and visual effects. It outperforms other state-of-the-art methods in terms of subjective and objective evaluations.


I. INTRODUCTION
With the development of computer science technology, medical imaging plays a vital role in the clinical diagnosis. Among them, Magnetic Resonance Imaging (MRI) images primarily depict soft tissues, such as blood vessels [1]. Computed Tomography (CT) images can clearly reflect the precise localization of dense structures [2]. It is nearly impossible to get them both from any single medical modality as they provide information from different aspects with their own advantages. Therefore, it is necessary to fuse MRI and CT images to meet the requirements of more complex diagnosis, for example, skull base tumor detecting. Medical image fusion has become a widely used tool for creating high-quality images with amounts of information in order to increase the capability The associate editor coordinating the review of this manuscript and approving it for publication was Shiqi Wang. of diagnosis of medical problems. It can combine multiple images from single or various imaging modalities into a fused one. Also, the fusion process can reduce the randomness and redundancy. At present, there are several main methods for medical image fusion, such as spatial domain based, transform domain based, neural network based, hybrid methods and others. We review them briefly.
In spatial domain, the pixels of images can be calculated directly. There are many techniques used widely, such as Principal Component Analysis (PCA), Averaging, Simple Maximum/Minimum method, Bovey Transform (BT), Intensity Hue Saturation (IHS), High Pass Filter (HPF). However, it may produce spatial distortion in the fused images when only using spatial domain methods. The hybrid methods [3]- [6] can improve the quality of the fusion results.
The source images can be decomposed to extract features in addition to being processed directly. For example, the methods based on Laplacian Pyramid (LP) with multiple features can improve the ability to present outline and contrast of the images [7], [8]. The algorithms using Guided filter (GFF) [9], [10] or Cross Bilateral Filter (CBF) [11] have a good edge preserving property. The multi-level local extrema (LE) schema [12] improves the quality of the fused image. Besides, the methods via multiscale analysis would have the ability to select the frequencies in both space and time because the details in images are usually in the high frequencies. Such as DWT [13]- [15], Curvelet Transform (CVT) [16], [17], Non-Subsampled Contourlet Transform (NSCT) [18]- [20], Non-Subsampled Shearlet Transform (NSST) [21], [22] and Daubechies complex Wavelet Transform [23]. But multiscale transformation methods cannot completely extract the effective information from the base layers of images, so the fusion effect is not ideal in some aspects. Meanwhile, some new image decomposition methods have been proposed for image fusion, such as ones based on Sparse Representation (SR) [24]- [27] and saliency detection [28]- [30].
The transform domain based fusion framework requires the process of image reconstruction, the inverse transformation. Relatively, the end-to-end image processing model is more concise. The methods using Convolutional Neural Networks (CNNs) [2], [31]- [34] to extract features become the research trend gradually. But its interpretability in medicine still needs improvement. There are still limitations of the complicated design of fusion rules for different layers [35]. Whereas the generative adversarial network-based methods can be generalized to fuse images with different modalities and avoid causing functional information blurring or texture detail loss [36], [37]. Besides, some researchers deal with image fusion problem as an optimization model [38]- [41], and use optimization algorithms to combine weighted sub frequency images adaptively. However, these methods have limitations, for example, the need for multiple iterations to find the optimal solution. Approximately, the image fusion process is shown in Fig. 1.
But the white boundary in (b) is not completely preserved. Besides, the tissue information in the center of (a) is blurred after the fusion. To overcome these shortcomings, a novel image fusion method for MRI and CT images is proposed in this paper. The framework is realized using ADE based model and the features in spatial domain. Compared with the basic scale-space filters above, the anisotropic diffusion is sensitive to texture, and performs well in preserving edge information. It can separate images into base layers and texture layers better [42], [43]. At the same time, the fusion rule for texture layers is designed using the common coefficients obtained by image decomposition. Therefore, the process of image decomposition and fusion has consistency in identifying textures which is beneficial to retaining more textural details. For base layers, we choose the ''Maximum Absolute Value'' rule. The contributions of this paper are four-fold: 1) We propose a preprocessing step for generating a ''common image'' in order to control the synchronism of the decomposition process of the two source images.
2) The fusion rule designed for texture layers is constructed using common coefficients obtained by decomposing the ''common image''.
3) We establish consistency check on the decision map to mitigate the step effects from anisotropic diffusion. 4) We modify the settings and selections of parameters in the proposed algorithm.
The rest of this paper is organized as follows. A survey of related works and motivation are given in Section II. In Section III, the proposed image fusion model is described in detail. The experimental results and analysis are displayed in Section IV. At last, Section V concludes this paper.

A. ANISOTROPIC DIFFUSION
The anisotropic diffusion [44] treats images as thermal fields, and each pixel as a heat flow. If the pixel (x, y) is significantly different from others in the neighborhood, it may be retained as boundary instead of being diffused. That is, the filter can smooth a uniform region of a given image. Meanwhile, we use a Partial Differential Equation (PDE) to process nonuniform regions, that is, boundaries [45], which is similar to the bilateral filter. The anisotropic diffusion equation is described as: where the divergence operator is indicated with div, ∇ and represent the Gradient operator and Laplacian operator (divergence of gradient), respectively. c(x, y, t) indicates the rate of diffusion, and t means iteration number. A lower resolution image I t can be obtained after the anisotropic diffusion function is executed on image I t−1 . The solution to Eq. (1) is where λ ∈ [0, 1/4]. I t x,y calculated from the previous one means the pixel (x, y) in image at t scale. The subscripts N , VOLUME 8, 2020 S, E and W are abbreviation for North South East and West. The symbol∇ means nearest-neighbor differences: In Eq. (2), c t−1 Nx,y c t−1 Sx,y c t−1 Ex,y and c t−1 Wx,y indicate the diffusion function c(x, y, t) in the corresponding directions, respectively. They are defined as: where c(x, y, t)= g ( ∇I (x, y, t) ) . The function g(·) will be discussed in Section III in detail.

B. SUM OF MODIFIED LAPLACIAN (SML)
The traditional SML algorithm is able to reflect the edge information of an image as well as the sharpness. But it only calculates the Laplacian value of each pixel in horizontal and vertical directions. SML is defined as: where step denotes a variable spacing between pixels. The accumulation window about calculating the value of ML is set to (2N + 1) × (2N + 1).

C. MOTIVATION OF THIS WORK
In image fusion, how much details are retained in the fused image largely determines the image quality, which will have an impact on clinical diagnostics. The combination of anisotropic diffusion and SML can separate details from the source image more accurately. It is not difficult to find that the anisotropic decomposition model and SML algorithm use similar ideas in the image calculation process. That is, the North South East and West are equivalent to vertical and horizontal in direction. But the selection of only two directions reduces the accuracy and it cannot make use of the strong correlation among adjacent pixels. Therefore, we desire to expand the selections and use more adjacent pixels for calculation. Besides, we need to use parameters to associate the decomposition process with the fusion method in order to improve the consistency in identifying textures of the overall framework. The acquisition of parameters depends on the process of image decomposition. Meanwhile, we hope that these parameters are obtained through calculating two source images, which makes the whole model more adaptive.
As mentioned above, inspired by some recent advances in anisotropic diffusion method [45], [46] and the importance in synchronized decomposition, the S-ADE image decomposition method (presented in Section III-A) is introduced to decompose medical images. The NSMAL algorithm (proposed in Section III-B.2) is designed to fuse texture layers of medical images. The relation between the prior works and proposed methods is shown in Fig. 2.

III. PROPOSED METHOD
In this section, we describe the proposed model for MRI/CT medical image fusion. Here, we assume that the source MRI and CT images are well registered. There is no need to consider about the problems about the registration. First, we generate a ''common image'' based on two source images. We can get the common decomposition coefficients (direction parameters) by decomposing the ''common image''. After image decomposition via S-ADE, the base layer and texture layer of each image are obtained. Then fuse the base layers using ''Maximum Absolute Value'' method. The texture layers are fused by NSMAL algorithm. At last, the fusion result can be obtained through the image reconstruction. The framework of the proposed model is presented in Fig. 3.

A. IMAGE DECOMPOSITION USING S-ADE 1) E-ADE
As mentioned in the motivation, the traditional ADE model can be expanded. In detail, the four nearest points around the pixel (x, y) are in the N , S, E and W directions in image. The second nearest points are in the Northwest (NW ) Northeast (NE) Southwest (SW ) and Southeast (SE) directions. So there are eight points contributing to calculate the energy of the pixel (x, y). The Eqs. (3) and (4) can be extended to the following:∇

2) SYNCHRONICITY ESTABLISHMENT
The ADE image decomposition model can be adaptively performed according to the properties of the image itself. So the decomposition bases of different images are different in process. But image fusion is to combine salient features of multimodality images together, and express it through a single image. We hope that the source images are decomposed under the same condition to meet the fusion consistency. In our opinion, the solution is to generate a ''common image'', based on which the decomposition coefficients of different source images are obtained. The preprocessing method should be simple and effective. There are three solutions to the problem: PCA algorithm, average filter and stitching two images. One experiment result about the preprocessing is shown in Fig. 4. From the figure, it is obvious that the texture information marked by red box in MRI image displayed in Fig. 4(a) is important. Hence, we need to preserve in the fused images. However, the details are partially lost in the fused images obtained using PCA algorithm and average filter. By contrast, the fused image using image stitching preserves more textural information. Extensive experiments are conducted to compare the performance of the three strategies, and the experimental results demonstrate that stitching method performs better than the other two ones. The stitching method is simple but keeping the original information. Furthermore, we need to generate a ''common image'' after each iteration diffusion process, which is tedious and time-consuming when using algorithm or filter. But it needs to generate only one initial ''common image'' using the stitching method. The diffusion iterations are executed based on it.

3) DIFFUSION COEFFICIENT FUNCTION
It needs to determine the diffusion coefficient function g(·) mentioned in Section II-A after obtaining the required ''common image''. g(·) is a monotonically decreasing function with g(0) = 1. There are many selections meeting the requirements. According to the suggestions in [44], we summarize as follows: where α > 0. The function privileges wide regions over the smaller ones. The constant K is called edge threshold, which is important to the function. If K is too large, the base layer information from CT image will be lost. On the contrary, the smooth diffusion on the MRI image stops earlier, resulting in insufficient texture information of the image after decomposition. In order to retain useful information of MRI and CT images effectively, we need to discuss the choices of K and α. Besides, as a non-linear iterative filter, the number of iterations of the S-ADE algorithm determines the degree of separation between texture layer and basic layer. Specifically, the details will be retained from MRI image enough if the number of iterations is large. By contrast, more basic information of CT image will be preserved. The choices of parameters for the diffusion function are shown in Section IV.

4) IMAGE DECOMPOSITION
The image I t x,y is decomposed by the S-ADE model for obtaining base layer: where B t I is the t-level base layer and S − ADE(I t x,y ) means that the proposed model decompose the t-level image. The texture layer can be obtained by the subtraction: In summary, the image decomposition method based on S-ADE model can be summarized as Algorithm 1. The ''common image'' is generated in Step 1. From Step 2 to 6, we get the ''common base layer'' B t I . After Step 7, the ''conmmon texture layer'' T I can be obtained. At last, we get the output layers through Step 8 and 9.
Two groups of results about decomposition on MR-T2 and CT images are shown in Fig. 5. From (e) and (f), it is clear that the textural details separated from MRI images are more than the ones from CT images, and the base layer information in (c) and (d) is well preserved. The results indicate that the proposed image decomposition method is sensitive to the texture information. That the base layer and texture layer are well separated can improve the fusion effect.

B. IMAGE FUSION RULES
The base layer and texture layer present different information, and they largely vary in significance for MRI and CT image. For example, texture information of MRI is more useful, and basic information from CT can reflect the bone structure better. Therefore, different fusion methods are designed for different layers.

1) BASE LAYER FUSION RULE
The fusion method for base layer is simpler as the detail information contained in B I is very limited. In this framework, we use the ''Maximum Absolute Value'' rule as the fusion algorithm. We get the fusion map using: In this way, the base layer B F of the fused image can be obtained by: The fused base layer results are shown in Fig. 6.

2) TEXTURE LAYER FUSION RULE
On texture layers, the fusion decision map is calculated by NSMAL algorithm which is designed using directional coefficients obtained by decomposing MRI and CT images. Similar to the E-ADE model, the traditional SML algorithm can be extended to NSML, and NML is defined as: In order to establish the consistency between the decomposition and fusion process, the NMAL is proposed as: where the weight coefficient ω > 1 as we hope to preserve more details from MRI image. In order to mitigate the staircase effects in fused texture layer, we establish a consistency check on the decision map map T through a sliding window, which aims to further improve the quality of fused images. The window is set to 3 × 3. Similar to the image decomposition process, we calculate the sum of eight values around the pixel in the map. If more than half of the pixels are from the MRI image, the value at pixel (x, y) is set to 1. By contrast, the value is set to 0 (see Eq. (17)). The fused texture layer can be obtained through Eq. (18).
The texture layer fusion results are shown in Fig. 7.

3) IMAGE RECONSTRUCTION
The image of low resolution which is treated as base layer is obtained using S-ADE model to decompose the source image. We subtract the low frequency information from the source image to get texture layer according to Eq. (11). The image reconstruction process is the inverse process of decomposition. Thus, the fused image F is given by a linear combination of base layer and texture layer as: Two groups of fusion results images are shown in Fig. 8.

4) IMAGE CORRECTION
There are gray lines within the white outline of the fused results by observing the fusion images displayed in Fig. 8. In our opinion, the lines are texture information from MRI image and the white outline is preserved from CT image. The white part of CT image reflects the bone information. But current fused images may mislead a diagnosis as a fracture. Therefore, we add a step to modify it. We get an intensity weight map of CT image: where θ is an intensity threshold used to extract contour from CT image. Pixels with an intensity greater than θ are retained in white outline of the fused image. The residual texture information of the MRI image in the white outline of fused image is eliminated by: The final fused images are shown in Fig. 9.  Larger values indicate more information and better fusion results. The metrics include:

IV. EXPERIMENTAL RESULTS AND ANALYSIS
(1) Mutual Information (MI) [47] measures the mutual dependence of two images which is defined as: where p(u, v) is the joint probability distribution function, p(u) and p(v) are the marginal distribution function. The quality of the fused result with respect to source images can be calculated as: (2) Universal Quality Index (Q 0 ) [48] models distortion as a combination of three factors: loss of correlation, luminance and contrast distortion. It is defined as: wherex andȳ mean the average, σ x and σ y represent the variance, σ xy = 1 (3) Edge-Based Similarity Measure (Q AB/F ) [49] measures the similarity between the fused image and source images by evaluating the amount of edge information transferred in the fusion process. The Q AB/F is defined as: where w X i,j is the weights for Q XF i,j , (X = A, B). The definition of Q XF i,j is given as Q XF presents the similarity of the width and direction between X and F at location (i, j).
(4) Nonlinear Correlation Information Entropy (NCIE) [50] has excellent mathematical properties as a measure for the nonlinear type of correlation of concerned variables: where λ R i (i = 1, . . . , K ) are the eigenvalues of the nonlinear correlation matrix R.
(5) Visual Information Fidelity Fusion (VIFF) [51]: where FVI D k,b and FVIN D k,b denote fusion visual information with distortion and fusion visual information without distortion in the bth block at kth sub-band, respectively. (6) Feature Similarity Index (FSIM) [52] focuses on lowlevel features. It calculates the similarity between images using phase congruency (PC) and gradient magnitude (GM) information: where S L (x) = S PC (x) · S G (x). The similarity measure for PC 1 (x) and PC 2 (x) is defined as S PC (x) = 2PC 1 (x)·PC 2 (x)+T 1 . 91342 VOLUME 8, 2020  (7) Information Fidelity Criterion (IFC) [53]: where C N k ,k denotes N k coefficients form the RF C k of the kth sub-band, and similarly for D N k ,k and s N k ,k . (8) Visual Saliency induced Index (VSI) [54] employed as a weighting function is closely related to image quality assessment which reflects the importance of a local region: where ] β , α and β are two parameters used to adjust the relative importance of visual saliency (VS), GM and chrominance features, where M and N denote channels of images. The subjective evaluation criteria are described in TABLE 1. We invite five experts to evaluate the proposed method with other methods subjectively. All images are displayed under the same conditions.

C. PARAMETERS SETTINGS
In order to make the S-ADE model more suitable for the decomposition of MRI and CT images, we discuss the settings of diffusion function parameters. To improve the computational efficiency of the model, we choose constants as hyperparameters. The experiments are performed on ten pairs of images. The objective evaluation about experimental results is shown in Fig. 11.
It is obvious that most scores rise and stabilize gradually from K = 0.5 to K = 20, then they slide down slightly except the IFC and Q AB/F in Fig. 11(a). IFC keeps rising and Q AB/F decreases. Both of them flatten gradually, that is, the slope decreases. Because the value of Q AB/F decreases more in CT image than it rises in MRI image, the average one goes down. So we decide that K = 20 in the S-ADE model after analyzing the objective quantitative evaluation.
In order to maintain the properties of the diffusion function, α is an odd number. It is set to 1 after analyzing the sensitivity of the indices to changes of α in Fig.11(b). IFC decreases more obviously than MI, VSI and Q AB/F which rise a little in α ∈ [1, 3]. VOLUME 8, 2020  The evaluation results in Fig.11(c) indicate that most of the indices rise up slowly before 20 iterations, decline until 30 iterations. Because the details of MRI image increase less than the lost basic information of CT image during the iteration process, the scores of FSIM and Q AB/F reduce a little, but not enough to affect the conclusion, the number of iterations = 30. Fig. 12 shows the effectiveness in establishing decomposition synchronism using stitching on ten pairs of images in terms of objective quantitative evaluation. The scores in   the figure have been normalized in the same way for the convenience of display. The MI, Q 0 and IFC get improved obviously, NCIE, VIFF and VSI rise a little, but Q AB/F and FSIM reduce slightly. Overall, generating ''common image'' can ameliorate the fusion effect.

D. COMPARATIVE ANALYSIS AND RESULTS DISCUSSION
In this subsection, we will show seven groups of experimental results about medical multi-modality images fusion.
It consists of four groups of MR-T2 and CT images (MR-T2/CT), a group of MR-T1 and CT images (MR-T1/CT), a group of MR-Gad and CT images (MR-Gad/CT) and a group of MR-PD and CT images (MR-PD/CT). The proposed method is compared with 13 fusion methods: CBF [11], CNN [31], CS-MCA [27], GFF [9], LE [12], LP-SR [24], MSSF [28], TSIF [29], VSMWLS [30], MSVD [55], CVT [17], ADKLT [45] and FPDE [46]. The parameters of the methods above are the default ones from the provided codes. VOLUME 8, 2020   The first group of MR-T2 image and CT image fusion results from 14 methods are shown in Fig. 13. First, we observe the information obtained from CT image, which highlights bone information. The information in (c) and (f) is lost a lot because CBF and GFF methods perform well in details preservation but bad in base information retention. The results MSVD, ADKLT and FPDE have low contrast as the fuse rule they use for base layers perform not well. Then we pay attention to the central of (a). It is obvious that CS-MCA, LP-SR, MSSF, TSIF and CVT have poor performance. Combining the above two aspects, there are relatively excellent fusion visual effects in CNN, LE and proposed method whereas VSMWLS method has slight color distortion in center. The texture details in the red box are enlarged in the upper right corner of each image. There is faint texture information in the white outline in (d) and (g) which may  lead to misdiagnosis as we explained in the previous section. Besides, (d) and (p) retain more detail structure than (g). The subjective evaluation results are shown in Fig. 14(a). The bold in TABLE 2 indicates the highest score in terms of objective evaluation. It is clear that the fused image obtained by the proposed method is better than others. VOLUME 8, 2020 The second group of MR-T2/CT fusion results are shown in Fig. 15 which reveals that most fused images preserve the outline information except for CBF, MSSF, MSVD, CVT, ADKLT and FPDE. It is because that multiscale transform based or filter based methods cannot completely extract the effective information from the base layers. In addition, the fuse rules used for base layers in these algorithms do not perform well. Observing the eyes, CNN, CS-MCA, LE, TSIF, VSMWLS and proposed method perform better. Further, the texture information marked by red and blue boxes is enlarged to the corresponding directions. The details lost in (d), (e), (j) and (g) successively increase, that is, CNN<CS-MCA<TSIF<LE. However, the proposed method can preserve more details as the anisotropic diffusion function is sensitive to the texture and the fuse algorithm using common coefficients has excellent performance. The ''Maximum Absolute Value'' rule extracts information from base layers effectively. The objective evaluation (in TABLE 3) and subjective evaluation (in Fig. 14(b)) indicate that the fused image obtained by proposed method performs better.
The third group of fused images are shown in Fig. 16. The result (c) has relatively poor performance in preserving the information from CT image. The white part, bone information is lost a lot. There is color distortion in (l), (m), (n) and (o) where the skeleton part appears gray in visual. The texture details in red box are enlarged. It is obvious that there is noise in LE and MSSF feels blurry. The lines in contours of CNN, LP-SR and VSMWLS may lead to misdiagnosis. By observing closely at the enlarged part, our method retains more details than CS-MCA and TSIF. The evaluation indices in TABLE 4 illustrate that GFF performs best in Q AB/F but the information at bottom in fused image is partly lost. The proposed algorithm retains effective information though it ranks second. In other aspects, our method performs better. The subjective evaluation in Fig. 14(c) indicates that the proposed method has stable fusion effect.
The fourth group of MR-T2/CT image fusion results are shown in Fig. 17. The result (m) has artifacts which reduce image quality in visual. The (l), (n) and (o) have lower contrast. Considering about the central part of the image, (c), (e), (f), (h), (i) and (j) lose information to varying degrees. The details in red box can be observed carefully. The skeleton information from CT image are retained better in proposed method, than CNN, LE and VSMWLS. The objective evaluation in TABLE 5 shows that LE gets the highest score in Q 0 . The proposed method is in second position. The subjective evaluation in Fig. 14(d) shows that the proposed algorithm has excellent performance in visual.
The average subjective evaluation scores on four groups of experiments are shown in Fig. 14(e). It is clear that the proposed method has relatively stable performance with good visual effects. Besides the MR-T2 images, which are used widely, there are other MRI images, such as MR-T1, MR-Gad and MR-PD. The fused images between them and CT images are shown in Fig. 18. It shows that the proposed method can perform well.
The statistical results about objective evaluation indices on 20 groups experimental data are concluded in TABLE 6. Except for the Q AB/F , the proposed method obtains the highest scores. For Q AB/F , GFF performs best, and our method takes the second place. In our opinion, the GFF method is good at preserving the edge information due to its principle of decomposing images. So it has better performance in Q AB/F but worse in Q 0 than our method. In general, the proposed method based on S-ADE model has the ability to fuse MRI images and CT images effectively, maintaining more useful information as well as ensuring the quality of the fused images.

E. COMPUTATIONAL EFFICIENCY ANALYSIS
All the algorithms are implemented using the MATLAB 2016b, the simulations are run on Dell Inspiron 3668 with Intel Core CPU 3.0 GHz and 8GB of RAM.
The computational efficiencies of methods compared above are concluded in TABLE 7. Among CNN, CS-MCA, GFF, LE and LP-SR methods, which have excellent performance in fused images, the CS-MCA method is a time-consuming one as shown in the table. The CNN model needs amounts of time to train medical images to improve the quality of fusion results. The LP-SR method uses a trained dictionary to realize fusion work. But the time of their preliminary work is not counted in the running time. The GFF has higher computational efficiency than our method, which is higher than LE method. The other fusion algorithms have shorter running time. However, they perform not well in fusion results. On the whole, the proposed fusion method has excellent fusion results with acceptable computational efficiency.

V. CONCLUSION
In this paper, we propose a medical image fusion method based on S-ADE model. First, the MRI images and CT images are decomposed into two layers: base layer and texture layer. Then, the ''Maximum Absolute Value'' rule is used for the base layer fusion. On the texture layer, we design the NSMAL fusion rule using common decomposition coefficients in order to retain more details. In this way, the image decomposition and fusion process have consistency in identifying textural information, which is beneficial to improve the quality of the fused image. Simultaneously, the consistency check using a sliding window is designed for decision map on texture layer to mitigate the staircase effect. Finally, correct the fusion results after image reconstruction. In the experiments, we select subjective evaluation and eight different objective evaluation metrics including MI, Q 0 , Q AB/F , NCIE, VIFF, FSIM, IFC and VSI. Compared with 13 excellent algorithms, the proposed method gets the highest scores except for Q AB/F . The Q AB/F uses the local metric to estimate the performance of the significant information from the source images in the fused one. The higher the Q AB/F value, the more the edge information of the fused image. The GFF method performs better because it belongs to a filtering algorithm that can maintain edges. Our method is in the second place with a small score gap. At the same time, the computational efficiency is acceptable. Overall, the experimental results demonstrate that the proposed method can maintain more information from the two source images and preserve the visual effect of the image compared with 13 fusion algorithms used widely.
In the future, we will work on the research about the diffusion function design and the combination of neural networks method to realize image fusion. At the same time, we are going to try to improve the computational efficiency. In addition, the situation of anatomical and function image fusion would be helpful for clinical diagnosis. The proposed method can be extended to fuse anatomical and function imaging modalities via some color space transform methods. The approach that avoids introducing structural and color distortion during the fusion process needs further study.