A Dual Model for Restoring Images Corrupted by Mixture of Additive and Multiplicative Noise

A critical challenge in image restoration is the presence of various types of noise. Meanwhile, noise detection is a crucial step in mixed noise removal. This paper tackles the challenge of restoring images corrupted by a mixture of additive Gaussian and multiplicative Gamma noise. In the proposed method, we integrate the noise detection process into a variational model using a dual formulation of a maximum a posteriori (MAP) estimator. The variational model consists of a novel adaptive fidelity term and a plugin-and-play regularization term. The fidelity term contains an adaptive weight that can automatically detect the noise types, levels, and pollution ways for each pixel. There is flexibility in choosing a plugin-and-play regularization term. For example, we can use a model-based regularizer or a deep learning-based regularizer. In addition, we present a splitting algorithm to minimize the proposed cost functional. This splitting technique enables us to transfer a mixed noise removing problem to several subproblems, including noise removal and detection. The noise detection process can be iteratively estimated by the proposed algorithm itself. Therefore, in the numerical experiments, the proposed model outperforms the existing Rudin-Osher-Fatemi (ROF), Aubert-Aujol (AA), BM3D, and deep learning-based single type denoiser. Experimental results show that the proposed model can remove noise more efficiently and better preserve details in images. Compared to the existing best-performing single type denoiser, on average, the improvements of PSNR values range from 0.33 dB to 0.81 dB under noise mixture ratios α = 0.4, 0.6.

The additive noise (AN) model is formulated as where f, u, n : Ω → R are mappings from the domain of the images to the intensities of the observed image, original image, and additive noise, respectively. Common additive noise includes white Gaussian noise, uniform noise, and impulse noise. Many methods have been developed to remove additive noise, including variational-based methods [7], [8], [9], boosting techniques for variational-based methods [10], [11], [12], [13], wavelet-based methods [18], nonparametric estimation [19], chronological techniques [20], [21], nonlocal methods [22], [23], [24], and deep learning-based algorithms [25], [26], [27], to name a few. Amongst these methods,  [7] model. It is worth noticing that there are many adaptive approaches. Thanh et al. [10] proposed an adaptive method based on the high-order total variation. A fast and adaptive regularization was proposed in [11]. More related methods can refer to [12], [13]. He et al. [18] proposed an automatical estimation of the noise levels based on maximum a posterior (MAP) estimation and wavelet generic Gaussian. Khmag et al. combined the clustered batches of noisy images and hidden Markov models (HMMs) to remove AWGN in [20]. Furthermore, they proposed an adaptive denoising framework based on second-generation wavelet domain using hidden Markov models (SGWD-HMMs) in [21]. BM3D [22] is a well-known nonlocal method and can obtain a good performance to restore images corrupted by the additive Gaussian noise. Recently, CNN deep learning-based method (e.g. [27]) was designed to remove AWGN, and could significantly improve the quality of the reconstructed images.
The multiplicative noise (MN) model is formulated as where v : Ω → R denotes multiplicative noise, which follows some standard distribution such as Poisson [28], [29], Gamma [14], and Nakagami distributions [30]. The Aubert-Aujol (AA) variational model [14] is well-known for the case where v follows a Gamma distribution. The AA model is conditionally convex. Although the existence of a minimizer can be proven by the variational method [14], many convex optimization algorithms can not be applicable to the AA model due to its non-convexity. Therefore, many studies have focused on the construct of a convex model. For example, using the exponential transformation through replacing u by e u , Huang et al. [31] and Jin et al. [32] converted the AA model to a convex variational model. Moreover, Jin et al. [32] provided some theoretical analysis. Ullah et al. derived a new data term under the assumption that the noise followed Nakagami distribution instead of Gamma distribution in [30]. Huang et al. [33] applied higher-order curvature variation to a convex model, which was superior to others in image edge and corner preserving. More variational models for multiplicative noise removal can refer to TABLE 1. in [33]. However, in reality, the noise type is not necessarily either additive or multiplicative. Rather, it can be a mixture of the same type [34], [35], [36], [37] or a mixture of these two types [38], [39], [40], [41], [42], [43]. Tackling these mixture models is even more challenging, and the methods mentioned above cannot straightforwardly solve this problem. Recently, several methods have emerged for this purpose. For a mixture of additive noise (MoAN), new approaches were for reducing a mixture of Gaussian-Gaussian noise [34], [35], and a mixture of Gaussian-impulse noise [35], [36], [37]. Especially, Wang et al. [35] proposed an adaptive algorithm named EM-CNN based on CNN deep learning-based algorithm. The algorithm in [35] combined the previous fidelity term [34] and CNN deep learning-based algorithm [27], where the fidelity term identifies the two different additive noise. It can achieve a desirable quality of the reconstructional images. For a mixture of additive and multiplicative noise (MoAMN), the model is usually formulated as where k 0 , k 1 are fixed constants, n, v are components of additive noise and multiplicative noise, respectively. Several methods have been developed to deal with a mixture of Poisson-Gaussian noise [38], [39], [40], to name a few. Thanh et al. [40] proposed a model based on total variation and applied a linear combination of log-likelihood functions of Poisson and Gaussian distributions through manual settings. For the removal of mixed Gaussian-Gamma noise, some works have also been raised. Chumchob et al. [41] proposed a variational model based on total variation (TV) and a linear combination of the fidelity terms in [7] and [32]. In their model, the parameter that balances the fidelity terms of the additive and multiplicative noise contributions was chosen empirically. Ullah et al. [42] proposed a new model with a linear combination of fractional-order total variational (FOTV), FoE image prior, and the data fidelity term in [7]. The parameters that balance the above three terms were also chosen empirically. As a result, the performance of these two variational models depended strongly on manual intervention. A similar method is in [43]. There are other recent reports on mixed Gaussian and Gamma noise removal techniques in [44], [45], [46]. This paper is devoted to restoring images that are corrupted by a mixture of additive Gaussian and multiplicative Gamma noise. The proposed approach provides a unified framework for a mixture of additive and multiplicative noise (MoAMN). The first challenge is the automatic detection of the noise type, level, and pollution way at each pixel. This paper provides a unified variational model with statistical parameters to discriminate against the distribution and noise level. Operator splitting is adopted to tackle this problem. The corresponding algorithm includes several iterative steps: noise removal, parameter estimation for noise type and level, and denoising adjustment.
The contribution of this paper is threefold: • This paper proposes a unified variational method to remove MoAMN. The weighting function in the proposed model is adaptive because it can be iteratively determined by the cost function itself. Additionally, the weighting function in our model plays the role of a detector for types and levels of noise. Moreover, this paper provides different interpretations of the weighting function with the EM and soft-max from statistics and convex optimization, respectively.
• The existence of a minimizer and the convergence of the algorithm are proved under some mild conditions. • The proposed model can integrate TV, BM3D, and DCNN regularizers into its corresponding algorithm. Significantly, the proposed method with DCNN regularization combines the variational models and deep learning using the pluginand-play method. The idea of the algorithm in this paper can 2 VOLUME 4, 2016 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Organization of the paper. The related works are reviewed in Section 2. Section 3 provides a process of deriving the proposed model, including mathematical assumptions, fidelity construction, and theoretical analysis. An algorithm is designed for the proposed method using splitting methods in Section 4. Section 5 presents some convergence analyses of the proposed algorithm. In Section 6, we present the numerical results. Section 7 shows the conclusion.

II. RELATED WORK A. METHODS OF REMOVING ADDITIVE NOISE (AN)
In image processing, there are some classical methods to remove additive noise (AN), such as ROF [7], BM3D [22], DCNN [27] and their variants. They all focused on removing additive Gaussian noise. Rudin et al. [7] proposed the ROF model where Ω |Du|dx is the total variation of u, and λ > 0 is a fixed parameter. The drawback of the ROF model is that it will create some artifacts. BM3D [22] is a nonlocal method that combines wavelet shrinkage and inter-patch correlation. However, the BM3D method may over-smooth the patches that do not have self-similarity properties in the image. Other nonlocal methods can refer to [23], [24]. To improve the quality of the reconstructed images, Zhang et al. [27] proposed the IRCNN algorithm that could deal with the model where x x x i , y y y i are N noisy-clean patch pairs. For space reasons, other methods to remove AWGN are no longer described here.

B. METHODS OF REMOVING MULTIPLICATIVE NOISE (MN)
AA model [14] is a well-known variational method to remove multiplicative Gamma noise. The mathematical expression is where λ > 0 is a fixed parameter. To convert the AA model to be convex, Huang et al. [31] proposed the following model Jin et al. [32] provided some theoretical analyses about the model (7). Under the hypothesis of Nakagami noise, Ullah et al. [30] changed the data term and proposed a new model as For page limitation, many other variational methods are omitted here, and one can refer to TABLE 1. in [33].

C. METHODS OF REMOVING A MIXTURE OF ADDITIVE NOISE
Wang et al. [35] proposed a model (EM-CNN) to remove a mixture of additive noise (MoAN), which can achieve better performance than the other existing methods like weighted K-SVD [34] and so on. The model combined the EM method and the IRCNN algorithm. The general model can be written as Ω}; µ is Lagrange multiplier; and η is a penalty parameter. Then they applied the alternating minimization scheme to get iterative formulas. Especially, the iterative formula of u is a problem of removing AWGN. They chose the IRCNN [27] to update u ν . In addition, the expression of H for the mixture Gaussian model can be expressed as where u(x), f (x) are clean and noisy images, respectively.

D. METHODS OF REMOVING A MIXTURE OF ADDITIVE AND MULTIPLICATIVE NOISE
Chumchob et al. [41] proposed a variational model to remove mixed additive Gaussian and multiplicative Gamma noise. Their mathematical model is where α 1 , α 2 > 0 are two parameters used to balance the fidelity terms of additive and multiplicative noise. These This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication.  parameters are often chosen empirically. Ullah et al. [42] proposed another variational model. The model is expressed as Set u = e w . (12) can be written as where µ, α, α 1 , α 2 , α 3 > 0 are parameters. In order to see the comparison of methods more intuitively, TABLE 1 summarizes the main methods mentioned above.
In the above models, the parameters that balance the fidelity terms are fixed. Therefore, these models cannot detect the noise type and level. Thus, an inadequate prior to the mixed noise would subvert the very foundation of the models such that they are not able to produce desirable reconstructions. To improve this situation, we propose an adaptive MoAMN denoiser in the next section.

III. THE PROPOSED METHOD
According to the literature survey, the existing variationalbased works on MoAMN removal depend strongly on manual intervention. To automatically update noise parameters, an adaptive variational model with selectable regularization is derived for restoring images corrupted by a mixture of additive Gaussian and multiplicative Gamma noise in this section. The adaptivity is accomplished by the weights in the fidelity term, which is derived by adopting a dual formulation of the MAP estimation. The regularization term can be chosen among TV, BM3D, and IRCNN regularizations.

A. THE NOISE MODEL
There are some assumptions in the following: A1 The images are corrupted by mixed Gaussian-Gamma noise with a ratio of α. In particular, the additive noise n follows Gaussian distribution with mean 0 and variance σ 2 . The multiplicative noise v follows Gamma distribution with mean 1 and shape parameter L. The parameters α, σ 2 , and L are unknown. A2 The noise is the realization of a random variable. A3 The original clean image follows a Gibbs prior distribution.
Throughout the paper, U , N , V , F denote the random variable of the pixels in the clean image, additive noise, multiplicative noise, and the observed image, respectively. Meanwhile, denote u, n, v, f as the corresponding samples of these random variables. P X (x) and p X (x) denote the cumulative distribution and probability density functions of random variable X , respectively.
Let A be the event that the noise is additive and its complementĀ be the event that the noise is multiplicative. The unknown mixture ratio α = P (A). Then, the forward problem in our model is represented as For a better understanding, FIGURE 1 shows three noisy images corrupted by additive Gaussian noise, multiplicative Gamma noise and mixed Gaussian-Gamma noise, separately. It is unknown for any pixel how likely the clean image is to be corrupted by an additive noise or a multiplicative noise. In other words, α is unknown.  By assumption A1, the probability density function of additive noise N at each pixel is and the probability density function of multiplicative noise V at each pixel is Here Γ is the well-known Γ function parameterized by L. According to A3, the probability density function of the latent image u is where T is the normalized parameter and φ is a given potential function. This Gibbs prior is widely used to model natural images, for example, in [14].

B. THE PROPOSED FIDELITY
In this subsection, an adaptive fidelity term is proposed to detect noise types and levels automatically for the mixed Gaussian-Gamma noise. There are two propositions regarding the probability density function of the additive and multiplicative noise in the MAP.
Proposition III.1 (additive noise) Assume F = U + N in which U and N are mutually independent. Let p 1 (n) be the probability density function of N ; then, Proposition III.2 (multiplicative noise, [14]) Assume F = U V in which U and V are mutually independent. Let p 2 (v) be the probability density function of V ; then, Let m, n be the row and column numbers of a clean image, respectively. The total number of pixels is denoted by N = m × n. The original image u u u = (u 0 , u 1 , ..., u N −1 ) and the observed image for a given f f f in terms of the MAP. By the Bayes' law and logarithm operation, the MAP can be converted to the following maximization problem Because the noise values of the pixels are assumed to be mutually independent, we obtain Furthermore, the prior is where C is the numbers of clicks in the graph representation of the prior. By plugging (21) and (22) into (20), the problem (20) is equal to the following minimization problem where Θ Θ Θ = (α, σ 2 , L) is a parameter vector of the distribution for noise. According to the law of total probability and Propositions III.1 and III.2, the problem (23) can be rewritten as Note that its first term is an ln-sum term, which is difficult to handle in the minimization problem. Next, we will expand it by employing the following result reported in [47], which is equivalent to the EM algorithm. where in Proposition III.3, the problem (24) can be converted to the following form Plug the probability density functions (15), (16) and (17) into the above formula. For ease of notation, denote w i1 = w i and w i2 = 1 − w i1 = 1 − w i . With some simplifications, the minimization problem can be written as It should be noted that there are some differences between the discrete model (27) in this paper and that in [35], which is recalled in part C of Section II. The fidelity terms in (27) and the formula (10) in [35] are different. The reason for the difference is that the mathematical problems are totally different. Specifically, the types of mixed noise in this paper are addition and multiplication, while those in [26] are only addition. Furthermore, the differences of those two models also make the corresponding algorithms different, shown in the next section.
The continuous version of the functional J in the problem (27) can be written as Therefore, the proposed model for removing the mixture of additive and multiplicative noise (MoAMN) can be written as min where , and X is a function space.

C. SOFT-MAX INTERPRETATION OF THE PROPOSED FIDELITY
Note that the weight w w w in the cost of the problem (27) can adaptively detect the noise type and level according to the noise parameters Θ Θ Θ. This section is devoted to interpreting the weight w w w from the viewpoint of convex optimization. Although J is not convex with respect to (u u u, Θ Θ Θ, w w w), it is convex with respect to the single variable u u u or w w w. We show that this non-convex problem can be derived by a dual method of convex functions. There are some definitions and propositions.
Definition III.1 (Soft-max) Given a vector y y y = (y 1 , y 2 , ..y M ), for all ε > 0, the soft-max operator is defined by It is easy to check that lim ε→0 max ε (y y y) = max{y y y}.
Proposition III.4 Let F ε (y y y) = max ε (y y y). Then for any fixed ε > 0, F ε (y y y) is convex with respect to y y y.
The proof of Proposition III.4 is deferred to Subsection S1 in the submitted supplementary material. Proposition III.6 For any fixed ε > 0, if F * ε is the Fenchel-Legendre transformation of the soft-max function F ε , then where The proof of Proposition III.6 is deferred to Subsection S2 in the submitted supplementary material.
The problem (26) can be derived from (24) by applying the Proposition III.6. Set M = 2, y y y = y y y In this procedure, the weight w w w is a dual variable that can be used to classify noise.

D. REGULARIZATION
There are many choices for the regularization term φ(u). For example, we can use p-norm (p 1), p-pseudo-norm (0 < p < 1) or 0 pseudo-norm [49] of the gradient of u, i.e. φ(u) = Ω |∇u(x)| p dx. To preserve textures, one may set φ(u) as detail-preserving [50] regularization or image patches-based nonlocal operators such as nonlocal TV [23], BM3D [22], and low rank [24]. Moreover, to use the nonlinear prior in nature images, CNN-based methods [27] may be integrated. This can be done by considering the variational of φ with respect to u as a nonlinear convolution neural network. In this paper, we will use TV [7], BM3D [22], and CNN [27] regularizations for comparison. For the modelbased regularizers such as TV and BM3D, we will prove the existence of a minimizer for the proposed model. For the data-driving regularizer such as CNN, we present some numerical results to demonstrate its performance.

E. EXISTENCE OF A MINIMIZER
In this section, we prove the existence of a minimizer of (28) by setting φ(u) = T V (u). In this case, X is the well-known BV space. Without loss of generality, let γ = 1.
Theorem III. 1 Assume that f ∈ L ∞ (Ω) and 0 < inf Ω f, sup Ω f < +∞, then there exists a minimizer of the problem (29) We defer the proof of the theorem to Subsection S3 in the submitted supplementary material.
In conclusion, there are three points to highlight in the proposed model. Firstly, we propose a new adaptive fidelity term to detect the noise types and levels automatically. Secondly, compared with TV and FOTV regularizations in [41], [42], [43], the proposed model with DCNN regularization can take advantage of the CNN deep learning algorithm to eminently learn images prior. That is, the adaptive fidelity term and the DCNN regularization work together to remove as much noise as possible while retaining much images' prior information. Thirdly, we prove the existence of a minimizer in Theorem III.1.
A block diagram in FIGURE 2 is shown in advance to demonstrate the main structure of the proposed model clearly. The related algorithm is introduced in the next section for more details.

IV. RELATED ALGORITHM
This section presents an algorithm for the variational model (29) using the splitting technique. The discretization schemes for the variational terms are standard. For integration, we use the rectangular formula. For the derivative that appeared in the TV regularization, we use a central difference scheme. For better representation, set a a a = {a 0 , a 1 , ..., Using these notations, (29) can be rewritten as Dual Proocess where The above problem can be decomposed into three subproblems according to the alternating minimization algorithm. Subproblem 1: Subproblem 2: Note that for giving w w w ν , Θ Θ Θ ν , Subproblem 1 is for updating the clean image u u u ν+1 . Under this circumstance, it can be regarded as a MoAMN denoiser. By employing the augmented Lagrange method, Subproblem 1 can be transformed into the following problem min u u u,d d d The corresponding augmented Lagrange function is where p is a Lagrange multiplier and r > 0 is a penalty parameter. By the ALM algorithm, problem (38) can be solved by the following inner iterative scheme where τ > 0 is a constant, and ν 1 = 0, 1, 2 · · · is an inner iteration number. By applying ADMM, (39) can be calculated by the following iterative scheme For u u u ν1+1 , u u u ν1+1 = arg min u u u λφ(u u u)+ < p p p ν1 , u u u > By simple calculation, (41) can be rewritten as where g g g ν1 = d d d ν1 − p p p ν1 r . Note that this is a Gaussian denoiser so that we can apply one of ROF [7], BM3D [22], and IRCNN [27] algorithms in the proposed method. For d d d ν1+1 , We only need to solve a cubic equation Once there is a converged u u u ν1+1 with the inner iteration scheme (40), we set u u u ν+1 = u u u ν1+1 to get the solution to the u u u-subproblem.
Updating w w w ν+1 , Θ Θ Θ ν+1 depends on the fidelity term, which is mainly derived by the dual method. So the process of updating w w w ν+1 , Θ Θ Θ ν+1 can be called a dual process. Now, for Subproblem 2, by using the first-order optimization condition of α and σ 2 , the closed-form solutions for α and σ 2 can be easily obtained as: Generally, it is difficult to calculate the exact parameter L in the Γ distribution. However, Subproblem 2 only requires an estimation of the parameter in the Γ distribution by given w w w ν . Thus, we can derive L ν+1 by employing the parameter estimation method for the Γ distribution [51].
By the first-order optimization condition of L, the above problem can be converted to where ψ(L) := d dL ln Γ(L). This equation can be rewritten as According to [51], [52], we can approximate L ν+1 as follows which is within 1.5% confidence. Subproblem 3 also has a closed-form solution where Above all, the proposed algorithm consists of a MoAMN denoiser and a dual process that can update noise parameters and weights with iteration steps. So the proposed method can be named an adaptive MoAMN denoiser.
b. Synthesis: choosing suitable fidelity by (44) based on the updated noise parameters and weight. The parameter r is set appropriately to adjust the level of correction of u u u.

V. CONVERGENCE ANALYSIS
In this section, we show that Algorithm 1 is an energy descent. Moreover, the inner iteration of the u u u−subproblem can converge to the minimizer of the subproblem (35).
Recalling the derivation of the proposed model in Section III, we use a dual formulation of the MAP estimation to Gaussian Denoiser Image Degradation derive the problem (24). Due to the existence of the ln-sum term, we use Proposition III.3 to obtain the problem (26). Then, the problem of the minimization of H is replaced by the problem of optimizing J . We have the following property of H and J .
Theorem V.1 [34] The functionals H and J have the same global minimizer with respect to (u u u * , θ θ θ * ) .
Proof The proof can be followed as Theorem 1 in [34].
The two following theorems are discussed under setting φ(u u u) = T V (u u u) and u u u in the discrete BV space.
Proof The proof can be followed as Theorem 1 in [53].
Theorem V.4 Assume that (σ 2 ) ν , L ν , w w w ν , λ, r, f f f , u u u * , u u u satisfies the same conditions as Theorem V.3, and let τ = r; then, the sequence u u u ν1 generated by the iteration scheme (40) converges to u u u * , i.e., lim ν1→+∞ u u u ν1 = u u u * .
Proof The proof can be followed as Theorem 2 in [53].

VI. NUMERICAL RESULTS
In this section, we present the performance of the proposed model on the test images corrupted by a mixture of additive Gaussian and multiplicative Gamma noise. Algorithm 1 with IRCNN regularization adopts a trained and plugin-and-play Gaussian denoiser in [27]. The training set is the same as [27]  There are two evaluation indicators, the peak signal to noise ratio (PSNR) and structural similarity index (SSIM), to measure the quality of the restored images. The PSNR is defined by P SN R(I, u) = 10 log 10 mn i ,j (u i ,j − I i ,j ) 2 , where I and u are the original image and restored image, respectively, and m and n are the row number and column number of the clean image's pixels, respectively. A larger PSNR value indicates a higher quality of the restored images.
SSIM [54] measures the structural similarity between two images. It is defined as where µ I , µ u , σ 2 I , σ 2 u , and σ Iu are the mean value of I, the mean value of u, the variance of I, the variance of u, and the covariance of I and u, respectively. The small constants c 1 , c 2 > 0 make the computation of the SSIM stable.
We compare the results obtained by the proposed method with those by the existing methods, including the most closely related ROF [7], AA [14], BM3D [22], and IRCNN [27] algorithms. In the following experiments, the restored results are presented under the mixture of additive and multiplicative noise. The restored results of the proposed model are obtained using different regularization terms, including TV, BM3D, and DCNN. In the experiments, the test images are corrupted with different mixture ratios α ∈ {0, 0.2, 0.4, 0.6, 0.8, 1}, different variances σ 2 = 0.005, 0.01 for Gaussian noise and different shape parameters L ∈ {10, 20, 30} for Gamma noise.
In order to demonstrate the effectiveness of the proposed model, there is a clear visualization of the restored images in FIGUREs 4 and 5. FIGURE 4 shows the restored images of image Barbara under the true noise parameters α = 0.2, σ 2 = 0.01, L = 10. In these figures, we zoom in for a part of the images. Firstly, the restored image by ROF method misses some details, as shown in FIGURE 4 (c). There are some speckles in the restored image by the AA method in FIGURE 4 (d). In comparison, the proposed method with TV regularization yields a better-restored image in FIGURE 4 (e), which not only removes noise more clearly but also remains much more information of the original image. Secondly, the restored image by the BM3D method in FIGURE 4 (f) is over-smoothing, but the proposed method with BM3D regularization in FIGURE 4 (g) retains more image information. Similar comparisons can be found between FIGURE 4 (h) and (i) for the IRCNN regularization. Similar results can be found in FIGURE 5 and more results (FIG-UREs S1, S2, S3) are available as supplementary material. In short, with the same regularization term, it is found that the images restored by the proposed model have better texture details and fewer impurities than the existing methods. To further show the effectiveness of the proposed method, There is just a little difference between the estimations of L and its true values except for α = 1. Because the proposed model would degrade into a single additive denoiser when α equals 1, and there is no multiplicative noise in this case. It is a challenging task to precisely estimate all the parameters owing to unknown types and pollution ways of noise. The estimations of σ 2 and α are not very good. In fact, the restoration performance mainly depends on the estimation of the weight w w w. The estimation of noise parameters can be rough since they are only used for noise classification.
The classification of noise is a key step of this approach. Thus, it is crucial to estimate the weight w w w accurately. Comparisons of the proposed method under known or unknown w w w are done on image Cameraman as shown in  This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication.     BM3D and IRCNN while removing noise efficiently. To see more details, we choose the same rectangle area for each image marked in red and enlarge them in FIGURE11. The structures of the tissues are observed to be clearly preserved in the restorations produced by the proposed method, but not the restored images obtained by BM3D and IRCNN. Examination of the removed noise presented in FIGURE 10 shows that the noise removed by BM3D and IRCNN is almost uniform everywhere, while the noise removed by our method can be inhomogeneous. This scenario is reasonable since the weighting function in the proposed algorithm can adaptively detect noise types and levels. Similar results can be found in FIGURE12 and FIGURE14, which show the denoising results of real CT images [46].

VII. CONCLUSION
In this paper, we presented a novel model to remove mixed additive and multiplicative noise at different ratios. The introduced noise type detection function is novel and has a statistical interpretation. Moreover, we showed that the weighting function for noise detection is a function in dual space from the convex optimization perspective. We proved the existence      of a minimizer for our variational model with TV regularization. To design an efficient algorithm, we adopted the popular splitting scheme to separate the proposed model into several easily solvable subproblems, enabling us to apply many good regularizers such as BM3D and DCNN. The experimental results showed that our method produces good results under mixed additive and multiplicative noise.
Note that accurate detection of a mixture of additive and multiplicative noise is challenging. In our method, the minimizer may not be unique because our mathematical model is non-convex with respect to both w and u. Moreover, it is well known that the EM algorithm is locally converging. Therefore, the performance of our method partly depends on the initial parameters, α, σ 2 , and L. It would be interesting to integrate some other methods to improve the presented model.     than those with known w w w, it is still better than other existing algorithms. This fact indicates that the proposed method is applicable.
To show the statistical information, the average PSNR and SSIM values of the total 10 test images are listed in TABLE 4. It is obvious to see that the numerical comparison shows that most of the average PSNR and SSIM values produced by the proposed model are higher than those of the existing methods with the same regularizers. The best-performing method is the proposed method with IRCNN regularization. However, it is worth noticing that there are some exceptions in TABLE 3 and 4, such as when α = 0 and α = 1. The reason is that the noise becomes a single multiplicative noise or additive noise in these two cases, and there will be no improvement for the proposed fidelity term in this scenario. If we set all elements of the weight w w w as 0 or 1, then the proposed fidelity will be degraded to ROF's or AA's, and the algorithm will produce similar results to those obtained by the existing methods. That is to say, to obtain results similar to those of the existing methods for removing single noise, it is necessary to choose appropriate initial values and model parameters.
To more clear visualization of the relationship between the PSNR values and the parameters, FIGUREs 9 and 10 show some PSNR curves in TABLE 4. Specifically, FIGURE 9 shows the relationship between the average PSNR values and the mixed ratio of additive and multiplicative noise in increasing order as α = 0 : 0.2 : 1 and the fixed noise parameters σ 2 = 0.005, L = 30. When α = 0, the difference between the PSNR values obtained by the proposed model and the ROF model with the same regularizer reaches maximum magni-tude. As the mixed ratio increases, the difference decreases. When α = 1, the advantage of the proposed model almost disappears, and the ROF model was even slightly better than the proposed model. This outcome is reasonable because the mixed noise becomes a single additive noise with a Gaussian distribution in this case. Furthermore, FIGURE 10 shows the relationship between the average PSNR values and the difference in the two noise variances under increasing L = 10 : 10 : 30 while fixing the mixed ratio α = 0.6 and variance of Gaussian distribution σ 2 = 0.05. One can see that as L increases, the advantage of the proposed model decreases, indicating that the difference between the variances in the Gaussian and Gamma distribution influences the validity of the proposed model. To illustrate the effectiveness of the proposed model on colour images, the proposed method tests on Lena(512×512×3) and House(256×256×3). Specifically, each channel is dealt with for ROF, AA, and the proposed method based on the TV regularizer separately, and then the colour channels are combined to obtain the corresponding restored colour image. For CBM3D and IRCNN, we apply them directly to the proposed model as regularizers and compare them with the corresponding original method. FIGURE 12 shows the restored images of the test colour image House and enlarges parts of the restored images. One can find that there are similar analysis results for the colour image House as the previous grey images. VOLUME TABLE S3 as supplementary material. It is easy to find that the proposed method can better preserve details and textures.
We apply the proposed method to real noisy data. FIG -URE 15 shows a real noisy ultrasound image downloaded from the website http://www.lib.dmu.edu.cn/database/csyx. jsp. The real ultrasound image contains noise with unknown types and levels. FIGURE 15 shows the comparison of the restored images by different algorithms, which contains the proposed method, BM3D and IRCNN algorithms. As observed from this figure, the proposed method can preserve more image textures than BM3D and IRCNN while removing noise efficiently. To see more details, we choose the same rectangle area for each image marked in red and enlarge them in FIGURE 16. The structures of the tissues are observed to be preserved in the restorations produced by the proposed method, but not the restored images obtained by BM3D and IRCNN. Examination of the removed noise presented in FIGURE 15 shows that the noise removed by BM3D and IRCNN is almost uniform everywhere, while the noise removed by the proposed method can be inhomogeneous. This scenario is reasonable since the weighting function in   [7], AA [14], BM3D [22], IRCNN [27]    the proposed algorithm can adaptively detect noise types and levels.
Similar results can be found in FIGUREs 17,18,19, and 20, which show the denoising results of real noisy CT images in [55].

VII. CONCLUSION
In this paper, a novel model is proposed to remove mixed additive and multiplicative noise at different ratios. The introduced noise type detection function is novel and has a statistical interpretation. Moreover, The weighting function for noise detection is a function in dual space from the convex optimization perspective. We prove the existence of a minimizer for the proposed variational model with TV regularization. To design an efficient algorithm, we apply the popular splitting scheme to separate the proposed model into several easily solvable subproblems, enabling the application of many good regularizers such as BM3D and IRCNN. Significantly, the proposed approach with IRCNN regularization combined the variational method to detect noise parameters       and CNN deep learning algorithm to learn images prior. The numerical experiments were done on ten test grey images, two colour images, and three real noisy medical images. According to the comparison of the restored images, the method in this paper can preserve more image details when removing mixed noise than the existing ROF, AA, BM3D, and IRCNN algorithms. For the most attractive cases α = 0.4 and α = 0.6, the biggest improvements of average PSNR values on 10 test grey images can reach about 0.81 dB and 0.62 dB respectively between the proposed approach with IRCNN regularization and the existing IRCNN algorithm. Note that accurate detection of a mixture of additive and multiplicative noise is challenging. Firstly, in the proposed method, the minimizer may not be unique because the mathematical model is non-convex with respect to both w w w and u u u. Secondly, the estimation of the noise parameters is not accurate enough. So the estimation influences the denoising performance.Thirdly, the update step on denoising in this paper involves a Gaussian denoiser and a fidelity selection. It may cause incomplete denoising. Fourthly, it is well-known that the EM algorithm is locally converging. Therefore, the performance of the proposed method partly depends on the initial parameters α, σ 2 , and L. In the future, a convex mathematical model could be developed, which can refer to the transformation way of the AA model into convex. A new splitting way may need to make a perfect combination of an additive denoiser and a multiplicative denoiser for the algorithm. In this case, the existing CNN deep learning algorithm for denoising multiplicative noise can be integrated into the proposed model to improve the performance. Some other trendy single denoisers also can be integrated to improve the presented model. In addition, one can design a CNN 16 VOLUME 4, 2016 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication.     architecture according to the proposed variational problem by unrolling technique, and this will let the denoising CNN have physical significance and interpretability. We leave these as future research.