Infrared and Visible Image Fusion Based on Gradient Transfer Optimization Model

To tackle the problem of partial loss of image details in infrared and visible image fusion, a gradient transfer optimization model is proposed for the fusion of infrared and visible images. Firstly, an adaptive image decomposition method is proposed based on coupled partial differential equation, the infrared image and the visible image are decomposed into base layer and detail layer to extract the high-brightness target and the details of the two images. Based on this superior information of infrared image and visible image, the optimization model is designed to obtain the fusion image obvious target and rich details. For the proposed optimization model, Alternating Direction Method of Multipliers (ADMM) is used to decompose the original model into sub-problems that are easy to solve and iteratively optimize to obtain the optimal solution. The introduction of control parameters makes the model more flexible in different situations, and retains the thermal radiation information of the infrared image and the detailed information of the visible image to the greatest extent. The fused image visual effects and performance indicators are improved. We completed the experiment using a public data set and analyzed the experimental results. The experimental results show that the proposed method can better preserve the clear target and texture information of infrared and visible images, and the fusion results are more accurate and comprehensive. The experiment results also indicate that our method performs well and achieves comparable metric values with the state-of-the-art methods.


I. INTRODUCTION
Image fusion is an enhancement technology that plays a very important role in computer vision and image processing. It aims to generate information-rich images by combining images obtained by different types of sensors for subsequent target detection or human eye observation, recognition, classification and the like. The infrared sensor imaging system can penetrate smoke, fog, snow, and has strong antiinterference ability. It also can work at night, but the infrared image only describes the thermal radiation intensity of the object and there are not considerable texture details. The visible light sensor imaging system has high resolution, rich information on the edge and texture of the target, and strong anti-interference ability. However, the image quality is poor at night and in low visibility conditions. The fusion of infrared and visible images provides more comprehensive information for various imaging conditions, and the fusion results play a The associate editor coordinating the review of this manuscript and approving it for publication was Sudhakar Radhakrishnan . key role in aviation detection, military reconnaissance, and security monitoring [1].
From table 1, the methods based on multi-scale transformation are the most widely studied methods in image fusion. But, inappropriate conversion methods can easily degrade performance [23], and the current fusion rules are mostly simple, so that interference information such as ''halo'' is introduced to the results [24]. In recent years, neural networks have been applied to the field of image fusion, such as using convolutional neural networks (CNN) to obtain image  VOLUME 8, 2020 features and reconstruct fused images, or build a new deep learning architecture for infrared and visible image fusion problems [23]. The method has good adaptability, fault tolerance and noise immunity, but designing an appropriate neural network and adjusting the corresponding parameters is still a challenging task [24]. Other techniques such as methods based on total variation [20], fuzzy theory [26] and entropy [22], which bring new horizons to the fusion of infrared and visible images. Among them, the fusion algorithms based on total variation model are gradually proposed. Total variation is a mathematical model that has achieved fruitful results in the field of image deblurring, such as Zhuo et al. [27] integrated a flash gradient into a maximuma-posteriori framework to recover a sharp image by combining a blurry image and a corresponding flash image. And Je et al. [28] employs disparity information in image segmentation, and estimates a PSF, and restores a latent image for each region. The main difference between our method and these two methods is application background. They apply to image deblurring, while we apply to image fusion. And Son and Zhang [35] proposed a new method for decomposing the image pairs to create a new base layer of the infrared image, which can have visual appearance similar to the denoised image. This algorithm made an outstanding contribution to the denoising theory. Whereas in our algorithm, in addition to denoising and decomposing algorithms, we also propose an optimization model to achieve image fusion by minimizing the optimization equation. In 2016, Ma et al. [20] proposed fusion of infrared and visible images based on total variation method. The main idea of the work is to retain the intensity information of the infrared image while retaining the detailed appearance of the visible light image. The fusion result is visually a sharpened infrared image. However, the author ignores part of the information of the visible image, which may result in loss of details in the image. Zhou et al. [29] proposed a method called targetaware fusion, which introduced the concept of inner product to strengthen the control of gradient modulus and gradient direction. The fusion result is good and the visual information is richer, however, multiple controls are not absolutely necessary and have an adverse effect on computational efficiency. Inspired by the above algorithms and based on the superior information of infrared image and visible image, this paper proposes to effectively utilize the difference to design an optimization model of two types of images. The brightness information retention item is designed to retain the general outline of the target brought by the thermal radiation information in the infrared image, and the texture information retention item is designed to more accurately retain the gradient information between the visible image and the infrared image. Using optimization model will avoid the cumbersome problems of designing too many fusion rules in traditional fusion method, and will not introduce unnecessary interference information to fused images. Compared with the neural network or the method based on representation, the design idea is simple and easier to understand and solve. The items in the model are independent of each other and the introduction of various control parameters makes the model more flexible in different situations. Specific effects could be exerted by changing or increasing or decreasing one item. With the flexibility, this paper presents the simplest fusion model, reducing the computational burden, speeding up the fusion, and improving the quality of the fusion.
The main contributions of this study are as follows: (1) In order to solve the problems of distortions and artifacts in fusion images, an adaptive image decomposition method is proposed based on coupled partial differential equations(CPDE). The coupling coefficient is introduced in CPDE to protect and enhance the edge while avoiding ''block effect''. The introduction of the CPDE decomposition method also can obtain good fusion effect even in the case of image degradation.
(2) We formulate the image fusion as an optimization problem characterized by introducing a brightness information retention item and a texture information retention item and seek the optimal fused image by minimizing an optimization function. Although our method needn't design troublesome fusion rules, it overcomes the problem of missing image details and also obtains a fusion image with a significant target and detailed appearance.
(3) Our method achieves remarkable fusion performance on the ''TNO Image Fusion Dataset''. By comparing with other algorithms, our fusion results have not only the best visual effect (the fusion images look like sharpened infrared images with clear highlighted targets and abundant textures), but also superior performance index.
The rest of this article is arranged as follows. In part 2, we present an optimization model for this algorithm and give a detailed solution for the model. In part 3, we show the experimental results and comparisons with other algorithms. In part 4, we summarize the proposed algorithm.

II. THE PROPOSED FUSION METHOD
In order to obtain more comprehensive information in the fused image, the proposed algorithm expresses the image fusion as an optimization problem. By iterative optimization of the designed objective function, a fused image is obtained which retains both the infrared image thermal radiation information and the visible image detailed appearance information. The framework of our proposed algorithm is shown in Fig.1. In this section, we comprehensively describe the proposed fusion algorithm. Fig.2 depicts a framework of the algorithm we proposed herein and related algorithms. The following algorithms are similar to ours: GTF [20], ALF [30], TAD-PGF [29]. Inspired by these articles, we proposed the algorithm of this paper. In order to clearly show the difference between our method and the above algorithms, we drew the following flow chart. The blue box in the figure represents the algorithm proposed in this paper. The red box is the GTF algorithm proposed by Ma et al. [20], and the green box is the ALF algorithm VOLUME 8, 2020  proposed by Guo et al. [30], and the purple box is the TAD-PGF algorithm proposed by Zhou et al. [29].
For readability to readers, the notation table of this article is listed in Table 2.

A. IMAGE DECOMPOSITION BASED ON COUPLED PARTIAL DIFFERENTIAL EQUATIONS
Due to the difference of image feature information between the infrared and the visible image, we can extract these significant information and analyze them separately, that is, separate the infrared target from the background, and extract the detail information in the visible image. This processing method can highlight thermal infrared targets while preserving image details. This method can be achieved by using the method of decomposing images. In this way, the original image is decomposed into a base layer and a detail layer, and then analyzed separately. For example, the fusion strategy of the base layer and the detail layer are formulated separately, and then the obtained base fusion layer and the detail fusion layer are merged, so as to utilize the source image information as much as possible. In the past work, the decomposition method based on anisotropic diffusion played an important role, such as the traditional anisotropic diffusion algorithm: (1) where c(|∇f |) = 1 1 + |∇f | 2 k . The above algorithm effectively preserves edge information while removing unnecessary noise. Anisotropic diffusion is a second-order partial differential equation, which essentially evolves the image toward the segmentation plane. Although it maintains a good effect on the edge of the image, it also causes the image to have a ''block effect'', the image looks like it covers a large mosaic, which adversely affects the subsequent process of the image. In order to overcome the ''block effect'' brought by the second-order partial differential equation, and take advantage of its good edge-preserving features, we proposed an adaptive decomposition strategy based on Coupled Partial Differential Equations(CPDE), it contains second-and fourth-order partial differential equations. Unlike the second-order partial differential equations, the fourth-order partial differential equations approach the original image by segmented slopes.
The area of the grayscale gradation does not transform the image into blocks with different gray values as in the secondorder equation. The fourth-order partial differential equations smooth it into a region of grayscale gradation, thereby eliminating the ''block effect''. So, it can avoid the above problems well, the generated image looks more natural, and the artifacts are reduced accordingly. In order to make full use of the advantages of fourth-order model and second-order model, a decomposition method proposed in this paper is as follow: where g is the coupling coefficient, m is the number of decomposition layers, and f 0 b is the input image f when m = 1. The corresponding second derivative is calculated as follows: The above decomposition method includes an adaptive decomposition process based on detecting image edges, the coupling coefficient g is selected as the edge detection function g = 1 1 + k|∇G σ * f | 2 , G σ represents a Gaussian function with variance σ . Gaussian filtering is performed on the image to smooth the effect of noise while protecting edge. At the edge of the image, the smaller g is, the larger 1 − g is, that is, the coefficient of the fourth-order term is smaller while the coefficient of the second-order term is larger, then, the second-order term plays a major role to protect and enhance the edge. Conversely, in the flat region of the image, the fourth-order term plays a major role, thereby avoiding ''block effect''. The decomposition algorithm proposed in this paper is an edge-preserving algorithm. Pixels with large changes in the gradient are regarded as edges, at this time, the second-order term plays a leading role and performs edge-preserving. Therefore, although the filtering operation is performed on the image first, the edge of the image is still maintained. The effect of the decomposition algorithm is shown in Fig.5. Fig.6 shows the edge retention ability of the decomposition algorithm, while Fig.7 shows the impact of the decomposition algorithm on the fused image. And performance index comparison of the fused image edge strength is shown in Table 3.

B. DESIGN OF FUSION IMAGE OPTIMIZATION MODEL
In this paper, we assume that the given infrared image is completely registered with the visible image. The infrared images, visible images and fused images are both grayscale images of the size m × n, with gray-scale values ranging from 0 to 255.
Inspired by Ma et al. [20] which aims to make the fused image look like a sharp infrared image with a clearly prominent target, we present a new fusion optimization model. The design idea of our model is to make the fusion image not only retains the intensity information of the infrared target, but also preserves the rich texture information of the visible image. Such a design can make the fusion result full use of the context information of the target to better accomplish the subsequent target detection and recognition tasks. Due to the obvious thermal radiation target of the infrared image, we expect the infrared image to control the overall contrast and target brightness of the fused image; at the same time, in order to reduce the loss of texture details, the texture of the fused image is obtained from the infrared and visible images. According to this idea, the proposed optimization model includes a contrast information retention item and a texture information retention item. The contrast information retention item constrains the fused image to have a pixel intensity similar to that of the infrared image. The texture information retention item controls the gradient distribution in the original infrared and visible images to be transferred into the fused image. The algorithm model in this paper is as follows:

1) DESIGN OF CONTRAST INFORMATION RETENTION ITEM
By observing a large number of images, the brightness information of the target is obvious in the infrared image, while that of the visible image is very weak and even difficult to observe the target in some images. In other words, infrared targets are easier to be detected from the background than visible light targets. Therefore, when retaining the overall contrast of the fusion image, we only consider constraining the base layer of infrared image. Then the contrast constraint can be expressed as follows: In the previous work, both L1 and L2 norm constraints appeared. But as pointed out in Reference [30], when the difference between the unknown fused image and the constrained target is Gaussian, it is best to use the L2 norm. The L1 norm specification will be more appropriate when it is a Laplace or pulse distribution. In our problem, most pixels are expected to have similar strength to the source image, with only a small portion being used to provide gradient information, in which case the L1 norm specification is better. Thus, we determine the luminance and contrast information retention term as equation (13):

2) DESIGN OF TEXTURE INFORMATION RETENTION ITEM
Our goal is to include more detail information than the source image. In this paper, we describe the detail information using texture features, while the texture features are often represented by the gradient of the image. Since in most cases, for a source image with a complicated scene, the detail information is mainly concentrated in the visible image, whereas the infrared image also has some texture, if only consider the texture information of the visible image, the details may be lost. Therefore, in order to reduce the loss of details and get as much detail information as possible for the fused image, we constrain the gradient of the visible and the infrared image at the same time in the design of the texture information retention item. We take the gradient of the infrared and visible image into account by integrating the gradient information of the above two to obtain an initial fusion gradient map. We control the gradient information and introduce the weight coefficient to control the proportion of the two in different situations: ω 1 , ω 2 is the weight coefficients of the infrared and visible light images, respectively. The weighting factors in this paper are defined as follows: Using the initial fusion gradient field obtained above as the target gradient [31], the gradient of the fused image u d is close to the target gradient: Then, we have basically realized the control of the gradient information of the fused image, but for the gradient information we only consider the gradient magnitude as the constraint. Gradient amplitude constraints may corrupt the pixel intensity distribution, which may result in the loss of small amounts of information or distortion of the original brightness value of the infrared target. Each pixel has its gradient amplitude and gradient direction. In the constraints above, we control the amplitude of the gradient. Next, we add the gradient direction to obtain more accurate constraint information. Inspired by [29], we use the inner product to further control the direction of the gradient to obtain a more accurate fusion result, which is expressed as follows: where ∇u d , ∇f 2d represents the inner product of the fused image and detail layer of the visible image, and it is used to constraints the gradient of the fused image. The inner product here can be expanded to: where |∇u d | is the gradient magnitude of the fused image, |∇f 2d | is the gradient magnitude of the visible image detail layer, and (θ ∇u d − θ ∇f 2d ) is the difference in the gradient direction of the two. That is, when the gradient information of the iterated fused image and the detail layer of the visible image is closer, the inner product will be larger. And even when the gradient information of the fused image obtained in this iteration is consistent with the gradient information of the detail layer of the visible image, this item is 1, and then the fused image obtains complete gradient information.
From the above, the texture information retention item can be written as: With the contrast information retention term constraining the pixel intensity, the texture information retention term constrains the gradient magnitude and direction. We get the final optimization function: (19) where λ is the control parameter and controls the balance between the two. Then we obtained the fused image. In order to further enhance the brightness of the target to facilitate later processing, we add the base layer of the infrared image to the fused image: As a result, we end up expressing the fusion problem as a minimization problem in order to obtain a fused image with significant targets and details. This model was inspired by the fusion models of Ma et al. [20] and Zhou et al. [29], and their models both achieved excellent results. The method of Ma et al. is based on the total variation denoising model, which directly constrains brightness and gradient of the image. The fusion result is visually a sharpened infrared image. But the author ignores part of the information of the visible image, which may result in loss of details in the image. Compared to the model proposed by Ma et al., we first decompose the image before constructing the model, and the constraints on the information increase, so that more accurate control of important information such as targets and textures is achieved. The method of Zhou et al. decomposed the image into base layer and detail layer, and introduced the concept of inner product to strengthen the control of gradient modulus and gradient direction. The fusion result has good visual effects, however, multiple controls are not absolutely necessary. Compared to the model proposed by Zhou et al., we applied the CPDE to decompose the image into base layer and detail layer, and at the same time the CPDE is also an edge-preserving filter for image denoising. So, our method is a joint denoising fusion algorithm. In the design of the model, the algorithm in this paper is more streamlined, reducing unnecessary constraints and reducing the computational burden. The model of Zhao et al. [18] also provided ideas for the proposed algorithm, after decomposing the source image, they enhanced the edges of the detail layer image and then fused, so they achieved excellent fusion results. In contrast to their method, the decomposition algorithm in this paper is an edge-preserving filter, so even though there is no enhanced edge, a good fusion result is obtained.
The detailed solution of the algorithm is given next.

C. OPTIMIZATION ITERATIVE METHOD BASED ON ALTERNATING DIRECTION MULTIPLIER METHOD
Alternating Direction Method of Multipliers (ADMM) is a computational framework for solving optimization problems [30]. The objective function of the original problem model is equivalently decomposed into several solvable subproblems, and then solve each sub-problem. Finally, the solution to the sub-problem offers the global solution of the original problem. In the algorithm of this paper, we use ADMM to tackle the issue. For the original problem (19), we first convert it into an optimization problem with equality constraints: min u,z,y After obtaining the above ADMM form, we continue to construct the augmented Lagrangian function: At this point, the solution to the original problem can be obtained. The equation (21) is decomposed into three subproblems and iterated continuously. In the process of solving each sub-problem, the remaining quantities are regarded as fixed values.
Perform a Fourier transform on equation (23): Thus, performing an inverse Fourier transform to get u k+1 where, z, p, y, a, b, c) ∂z If z < 0, z, p, y, a, b, c) ∂z Then we can get z k+1 : • p-update The update method of p is the same as z: z, p, y, a, b, c) ∂y This is the solution to the original optimization problem we have worked out. By iterating the above sub-problems to convergence, the optimal solution of the original problem, that is, the detail layer fused image u d , can be obtained. And the final fusion image can be obtained by u = u d + f 1b .

III. EXPERIMENTAL RESULTS AND ANALYSIS
In the above sections, we have introduced the model of the algorithm and its specific implementation process. This section will be devoted to the analysis of the advantages and disadvantages of the proposed method by comparing with other advanced algorithms.

A. EXPERIMENTAL ENVIRONMENT AND PARAMETER SELECTION
This experiment is implemented by MATLAB R2018a. The computer is configured with a main frequency of 3.30GHz, Intel Core CPU, 4GB of memory, and 64-bit win7 operating system.
There is a control parameter λ in equation (19). During the experiment, we found that the fusion effect will change according to the parameter λ. Fig.3 shows the change of performance indicators when different parameters are selected. As shown in the figure, when the values of λ are 0.2, 0.4, and 0.6, respectively, the values of the performance indicators gradually increase, but are lower than the values of λ = 0.8. When λ = 0.8, each performance indicator shows an optimal trend. When λ is 1.0 and 1.2, the values change slightly and tend to be stable. At the same time, we observed the fused images obtained through different parameters, a part of fusion results is shown in Fig.4. When we set λ = 0.2, the fusion result is severely distorted, which only reflects the approximate outline of the fused image, but the target edges are blurred, and a lot of image details are lost, the texture of the fused image cannot be presented. When the value of λ is gradually increased, it can be seen from fig.4(b) that the fusion result is gradually clearer, the texture retained by the image is gradually increased, the contrast is also improved. As shown in fig.4(c), the fused image has clear targets and textures when λ = 0.8, and the visual effect of the image is more in line with the observation of the human eye. As λ continues to increase, the fused image gradient is more obvious, as shown in fig.4(d), the fused image is visually like an over-sharpened image. After compared all the values of λ, we found λ = 0.8 is the best. The fused image not only has good visual effect, but also has well performance indicators. So λ = 0.8 is finally selected, and it kept unchanged for all the images to be fused.

B. EXPERIMENTAL RESULTS AND COMPARISON
We use a common data set to test the proposed algorithm. The dataset used in the experiments is the TNO Image Fusion Dataset, which contains registered infraredvisible image pairs in various conditions. The performance of the algorithm is tested by direct observation and comparison with seven excellent fusion algorithms, they are DenseFuse [9], MDLatLRR [32], VSM-WLS [2], SAF [33], CSR [6], FPDE [19], GTF [20].

1) DECOMPOSITION ALGORITHM RESULTS
• Visual quality Using the above decomposition method, we decomposed the infrared image and the visible image into the base layer and the detail layer to perform the next fusion operation more effectively. The decomposed images are shown in Fig.5. We obtained the base layer f b   It can be seen from the Fig.5 that the base layer images obtained by the two algorithms are significantly different. The images obtained by the algorithm in this paper do not appear ''black shadow'' and the transition is natural, whereas the partial region in (d) has stratification. Figure 6 shows the edge retention capability of the algorithm in this paper. Figures (a) to (h) are the fusion images obtained by other advanced algorithms and our algorithm. We have intercepted and enlarged important information in the picture (such as the edges of the character and some texture information). The figure shows that the fusion image (d) of the SAF algorithm, the target person has the highest brightness and the most obvious edges, but the texture of the fence and road is not clear. It can be seen that the algorithm makes full use of infrared information. Image (b),(f) and (g) processed by MDLatLRR, GTF and FPDE respectively have a low brightness, and the edges of the characters are not obvious. From the images (c) and (h) we can see, the image information obtained by VSM-WLS and the algorithm proposed has high brightness and clear edges, and the texture of fences and roads is more obvious. It can be seen that although the algorithm in this paper performs filtering on the image, it still retains the edge information of the image. Fig.7. shows the effect of the decomposition algorithm on the fusion results. The first row is the fusion result obtained without decomposing and recombining, and the second row is the image obtained by decomposing • Performance indicators In this text, we measured some objective parameters to evaluate the fusion results. We quantitatively evaluated the performances of different fusion methods using four metrics, i.e., entropy(EN), structural similarity index measure (SSIM) [34], mutual information (MI) [34], and edge intensity(EIN). EN reflects the amount of information contained in the image. The larger the entropy value in the image, the richer the image information contains. SSIM is used to measure the level of similarity between images. The higher the value of the structural similarity, the higher the similarity between the fused image and the corresponding source image reveals. The higher the EIN, the sharper the edges of the image. Fig.8 shows a histogram of the results of this experiment, in which the yellow column represents the experimental parameters of the fused image after using the decomposition algorithm, and the green one is the parameters of the image obtained by directly fusion. It can be seen from the histogram that under the premise of using the optimization model of this paper, the performance indicators of the fused image obtained by using the decomposition algorithm is slightly better than that of the directly fused image.

2) FUSION RESULTS
• Visual quality Some samples of the images obtained from the experiment are shown in Fig.9, and from left to right are the results on the eight image pairs of Nato-camp, Kayak, Duine, Steamboat, Lake, Street, Kaptein, Road. We visually observed each set of test images, and the algorithms involved showed their excellent performance. In general, the details of the fusion image based on the Densefuse algorithm are clear, but the overall tone are heavier, especially the ''Duine''. The images obtained by the VSM-WLS and SAF algorithm have large image contrast, the thermal radiation targets have high brightness; but ''black shadows'' appear in some areas in images fused using SAF methods, for example, in image ''Nato-camp'', the transition at the fence is not natural, and this phenomenon also appears in the image ''Lake''. The details of the fused images obtained by the FPDE method are not much lost, but the infrared targets are not significant. The fused images based on the CSR method has better visual effects, with obvious targets and clear textures. The GTF results tend to preserve the thermal radiation distribution of the infrared images, so the infrared target brightness is suitable, but the texture information needs to be further enriched, for example, the image ''Lake'' misses texture information.  The last row is the algorithm proposed in this paper. It can be seen by observing the images ''Nato-camp'', ''Street'' and ''Road'', the contour of the object in the image is clear, and image ''Kayak'' and ''Lake'' show that the infrared target is significant; the design goal is basically realized. The experiment achieved satisfactory result.
In general, the images obtained by DenseFuse, VSM-WLS, CSR and ours have better visual effects, retaining important information of infrared and visible images. However, DenseFuse and CSR need to build a network and train it, and VSM-WLS needs to design a fusion strategy. It can be seen that the advantages of the algorithm in this paper are simple design and easy to understand. Figures 10 and 11 show two representative fusion examples in our experiments. For better comparison, some of the focus areas in the source image and the fused image are labeled with a rectangular box. Fig. 10 shows the fusion of the different methods for the ''Kayak'' source image. The scenes described contain a wealth of content, including beaches, pedestrians, ships, clouds and so on. Infrared images mainly capture the thermal radiation information of pedestrians and ships, whereas visible images extract most of the spatial details of beaches and clouds. In this example, the ideal fusion result is that the characters are prominent and the background information is obvious. The fused images based on SAF and FPDE have clear texture details, amd the texture of clouds and beaches is obvious. However, the high contrast of these two methods is also not friendly to visual observation, and appear visual artifacts (such as the sky and the sea in Fig.10(d)(f)). The fused image based on MDLatLRR and GTF methods have relatively low performance in the extraction of thermal radiation information from infrared images, resulting in low brightness for pedestrians and ship targets (pedestrians in Fig.10(b)(g)). The fusion of the DenseFuse method have high quality with clear texture and infrared targets. It can be seen from Fig.10(h) that the proposed method achieves higher performance in detail extraction (such as the stripes on the beach, clouds in the sky and coastline), and the heat radiation target has a higher brightness (such as pedestrians and ships). Fig. 11 shows the fusion results of the different methods for the ''Lake'' source image. The scenes describe information such as lakes, trees, grassland and so on. The SAF-and GTFbased methods are not well completed in this example, the texture of the grassland area in the fused images is not clear(see the fence in Fig.11(d)), and in addition, some of the background areas in the fusion result appear black, which also introduces an unnatural visual experience.The DenseFuse-,SVM-WLS-based and proposed methods have achieved a good visual experience. The lake is clear and the background information is rich. The method proposed in this paper is superior to retain the details of the visible image, and the texture is clearer than other methods (such as the trees and grassland in Fig.11(h)). Finally, we add Gaussian noise to the source image to verify the stability of our algorithm when the image environment is poor. The experimental results are shown in Fig.12. It can be seen from the Fig.12 that despite adding noise to the source images, the fused images still obtain clearer target and texture. It shows that our algorithm can complete the fusion work even when the image quality is poor.
• Performance indicators Observed assessment is more subjective, we also measured some objective parameters to evaluate the fusion results. We quantitatively evaluated the performances of different fusion methods using four metrics mentioned above, that is, EN, SSIM [34], MI [34], and EIN. According to the above four indicators, the image with higher value is better in our test. The experimental results are shown in Table 3 to Table 6. The red numbers indicate that our method performs best; the blue numbers indicate that our result is second only to the first, and the black bold numbers indicate that the relevant method is optimal. It can be seen that the indicators measured by the algorithm proposed in this paper are not optimal for each item, but the overall effect is good and the performance is relatively stable. Among them, the EIN and SSIM of the fused image are better, it shows that the algorithm in this paper retains more source image information and has stronger edge retention ability. It is worth mentioning that DenseFuse also shows extraordinary results, but its framework is cumbersome and the calculation process is complicated. It is not as easy to calculate and understand as the method of this paper. At the same time, we used the image fusion data set ''Nato-camp'' as a test sequence to verify the versatility of the method, which contains 32 pairs of infrared and visible images. The quantitative results of the test are shown in Fig.13, where our method is represented by a    yellow-brown star. The results show that our method is optimal for EN and SSIM. In most cases MI performs well. This shows that our method is generally effective, not only retains important target information in the infrared image, but also retains rich details of the visible image.

IV. CONCLUSION
In this paper, an algorithm for infrared and visible image fusion based on gradient transfer optimization model is proposed. Firstly, a decomposition algorithm based on the coupled partial differential equation is used to decompose the source images, and then we build an optimization model and solve it to obtain a fused image with significant target contour and rich details information. Since the control parameter between the brightness contrast information holding item and the texture information holding item is introduced, the fused image with different effects can be obtained by increasing or decreasing the parameter to control the proportion of the infrared and visible images, and it greatly expands the application scenario of fused images. Experimental results show that the proposed method gives a performance comparable with state-of-the-art fusion approaches, compared with the other eight fusion methods, the visual effects and performance parameters show that the proposed algorithm not only has good visual effects, but also has superior performance indicators.