Cartoon and Texture Image Decomposition Driven by Weighted Curvature

In this paper, we propose a curvature-guided model with divergence-free constraints to facilitate image decomposition. Since basic TV regularization has difficulty in processing edge geometry information of cartoon image, in order to preserve the edge features, we introduce a level set curvature term to smooth the uniform area, and use the edge indicator function as the weighted regularization to preserve the edge. In addition, to control the smoothness, a gradient function is proposed to balance the edge indicator function and the curvature term. On the other hand, we find that the existing partial decomposition models measure the oscillation function with the $H^{-1}$ functional, but this functional ignores the divergence-free vector field during the Hodge decomposition process, then the texture will lose some vector direction information, and then affects the decomposition results. By analyzing the theory of divergence-free vector field, a new decomposition model with the constraint of divergence-free vector field is proposed in this paper. Numerical experiments show that the proposed model can well preserves the edges of cartoon and protects the texture of repetitive patterns.


I. INTRODUCTION
Image decomposition is a fundamental and important research topic in the field of image processing and computer graphics. Image decomposition technology is to extract the main features and useful information of images by designing models and algorithms. The main goal of image decomposition is to decompose a given image f into two meaningful parts. i.e., f = u + v. One part is u, named as cartoon or structural component, contains the main geometric features of the image and it's a simplified approximation of the original image. The other part is v referred to as texture and consists of oscillating component and some fine details in the image. Image decomposition can extract useful information from images, so it has a wide range of applications in image compression [1], image inpainting [2], image fusion [3], etc. Generally, the methods to solve decomposition models can be divided into two main categories, i.e., model-based methods and learning-based methods. However, due to the diversity and complexity of images, the unified network structure The associate editor coordinating the review of this manuscript and approving it for publication was Shenghong Li. cannot produce a consistent optimal result for a large number of images. In the period of network training, this problem appears as the performance instability on individual inputs. That is, although the averaged loss of all training samples is well reduced, for a single image, the loss may increase as the training continues. Because of this performance instability, we choose to use the model-based methods for image decomposition.
The first variational model of image decomposition was proposed by Rudin et al. (ROF) [4] for image denoising. They used the total variational (TV) norm to characterize the cartoon component and the L 2 norms to characterize the texture component, so it can well remove the noise and preserve sharp edges. However, some texture details are destroyed. Therefore, Meyer (BV, G) [5] used the G norm which is weaker than the L 2 norm to describe the texture component, and the proposed model opened up a new path for image decomposition. Whereas, it is difficult to find its numerical solution because the Euler-Lagrange equation of this model cannot be obtained directly. Later, in order to improve this drawback, a large number of studies are inclined to the effective approximation of this model. Vese and Osher (VO) [6] first proposed the approximate model. They used L P (P ≥ 1) norm to approach L ∞ norm approximation (BV, G) model and solved it with semi-implicit finite difference method. To simply compute, Osher et al. (OSV) [7] took the situation in VO model to obtain another model called OSV model. This model uses the H −1 functional to measure the oscillation component, and it is easy to solve since the corresponding Euler-Lagrange equation is a fourth-order partial differential equation. In order to preserve the edges and contours of the image, Aujol and Chambolle [8] described texture with negative Sobolev norm, and gave another approximation form of (BV, G) model. Xu et al. [9] proposed a model using adaptive second-order total variation to extract texture features. Tang et al. [10] combined non-convex functions with the H −1 functional and proposed a variational decomposition NNTV-H −1 model.
As is well-known, TV regularization preserves the edges, but the staircase effect is often inevitable. To solve this problem, Chan et al. [11] used high-order derivatives in the model to improve image decomposition quality. Chen et al. [12] replaced the TV regular term in the ROF model with the adaptive regular term, and the proposed model could well avoid the staircase effect. Other decomposition models will be discussed in references [13]- [36]. For example, nonlinear filtering and structure-aware filtering were studied in [13]- [15]. Sparse representation [16] and morphological component analysis [17] were developed. Partial differential equation methods were studied in [18]- [21]. Multiscale decomposition methods were proposed in [22]- [24]. In addition, Tang and He [25] used G space to measure oscillation function and their model could capture oscillation components in the image. To improve decompose results, weighted Hilbert norm and block nuclear norm were introduced to the decomposition model [26], [27]; Ng et al. [28] combined decomposition and restoration technology to deal with blurred images. Alexander and Gupta et al. introduced decomposition methods driven by level set curvature rate and smoothing spline interpolation [29], [30]. In recent years, deep convolutional net-works for image decomposition have been emerging in [31], [32]. Other applications based on image decomposition can be seen in [33], [34].
In image decomposition, there is great difficulty in processing edge geometry information of cartoon. In order to preserve image edges and contours, many models were improved by modifying the regular terms, see in [28], [29] and [36]. Because curvature leads to developable surface which can preserve edge information well, Alexander and Pierre [29] conducted a simple modification of ROF, and they added a filtered curvature term to the regularization part of ROF model. Inspired by [29], we introduce a curvature term to the regular term to enhance the performance of the model.
On the other hand, the H −1 functional was used to measure the oscillation component, and it can effectively extract texture from the input image. However, the H −1 functional neglected a vector function obtained by Hodge decomposition. Mathematically, if the vector information of this vector function is neglected, the optimal solution of the model will be biased, and then affects the decomposition results.
In summary, we mainly do the following work to achieve high quality cartoon-texture image decomposition. First of all, we add the level set curvature of low-pass filtering to the regular term to filter out the high-frequency parts of the image. Meanwhile, a gradient function is added to the curvature term to balance the TV regular term and the curvature term. Secondly, the proposed model contains a divergence-free vector in Hodge decomposition, which ensures the accuracy of the vector field corresponding to the texture. Finally, the alternating direction method of multipliers and numerical experiments are designed to verify the effectiveness of the proposed method.
The rest of this paper is organized as follows. In Section II, we review some prior related works. In Section III, we explain the technical background associated with our work, and then discuss the proposed new model and the corresponding numerical algorithm. Section IV will discuss the choice of some parameters, and do some numerical experiments to show that our method is competitive with the state-of-the-art image decomposition works. Finally, some conclusions will be given in Section V.

II. RELATED WORKS
A classical variational model in the field of image decomposition is the TV regular model [4]. The model is as follows where λ is a non-negative parameter, f is the input image, u is the restored denoising image or the cartoon part, and is the bounded image domain. This model can not only remove noise but also protect the image edges. However, it does have some limitations, such as some texture details are destroyed. In order to improve the ROF model, Meyer [5] proposed to replace L 2 norm with G norm. The image decomposition model proposed by Meyer is as follows where λ is a positive parameter, u is the structure part, and v is the texture part. Although model (2) can well represent the texture, the particularity of G function space makes the model difficult to solve. To solve model (2), Vese and Osher [6] proposed an approximate model of (2) where λ, µ are balance parameters. The first term insures that u ∈ BV , the second term insures that f = u + divg, and the third item is the penalty item of v = divg. When λ → ∞, and P → ∞, then model (3) is the approximation of (2), and it was calculated by finite difference method.
Later, Osher, Sole and Vese [7] replace G space with H −1 space. For (3), take P = 2, v = divg, then the simplifycation of model (3) is obtained, called OSV model where λ is a positive parameter, is the Laplace operator, and −1 the inverse operator of the Laplace operator.

III. NEW MODEL AND ALGORITHM A. THE PROPOSED MODEL
In this subsection, we introduce some concepts and functions that will be used in the proposed model.

1) CURVATURE TERM
A level set curvature termk is added into the regularization part in [29] to get a more competitive decomposition result, wherek is obtained by applying a low-pass filter to the level set curvaturek of the input image f . The reasons for adding the curvature term are as follows: for a given image, the smooth region changes gently, so the gradient is small, instead, the gradient along the edges is large. However, at the corner of image, although the gradient is very large, its curvature is much larger than its gradient. Due to the large gap, the general TV regularization term cannot handle the image corner well. Inspired by this, our model adds the curvature term to the regularization term as compensation.
In order to make the low-frequency cartoon well decomposed from the given image, we adopt a Butterworth low-pass filter to remove the high-frequency content of the level set curvature imagek (f ).

2) DIVERGENCE-FREE VECTOR
It is worth noting that during the derivation of the OSV model (4), the Hodge decomposition of g is g = ∇p + q, where p is a scalar function and q is a divergence-free vector function, i.e., divq = 0. Take divergence on both sides of . However, OSV model (4) neglected q from the expression of g and using L 2 -norm to measure |g|, i.e., The optimal solution of (4) will be inaccurate if cartoon u has deviation, and g is obviously related to the cartoon u in (4), and then g is inaccurate, which will further affect the texture. In Fig. 1, the modulus and direction of g in Fig.1(a) are equal to ∇p, while g in Fig.1(b) are the sum of vector ∇p and q. According to the triangle rule, we can observe that neglecting q makes the vector information of g inaccurate, and then it will affect cartoon and texture. Therefore, to get more correct and complete texture, the proposed model preserves the divergence-free vector field.

3) CURVATURE-GUIDED MODEL
Vese and Osher [6] and Ng et al. [28] discussed the properties of their models when g belongs to L P (P = 1, 2, ∞) space, they found that the difference between the decomposed results with P = 1 and P > 1 is not significant, so we let g belong to L 1 space in this paper.
Based on the above analysis, the decomposition model with curvature term and divergence-free vector constraints is given as follows where λ is a tuning parameter, −1 the inverse operator of the Laplace operator, q is the divergence-free vector field. For convenience, the structural obtained from the decomposition of (5) is abbreviated to be u 1 , and the curvature of the level set of u 1 after low-pass filtering is written as k 1 .

4) ADAPTIVE WEIGHTED TV
Since the general TV model adopts the same penalty way for the gradient regular term, it is unfavorable in edge and detail preservation [37]. Therefore, we give an adaptive weighting function c (x) to the gradient regular term in (5). For c (x), we hope that it is small for where the gradient is large, and large for the regions with a small gradient. Specifically, we define c (x) is inspired by a work [38], which is defined as follows where K is a positive constant, G σ (x) is Gaussian convolution.
On the other hand, since the penalty imposed by the weight function c (x) is not enough to smooth the homogeneous regions, we appropriately increase the power h (x) of |∇u| in (5). The curvature k 1 in the high frequency region of u 1 is large. For this reason, the value of |k − k 1 | is relatively small, so we add less penalty, i.e., set h (x) is smaller. In contrast, h (x) is larger in the low frequency region [29]. Firstly, let where t is a positive constant, c (x) is to ensure that h (x) is small enough at the edges and boundaries. VOLUME 9, 2021

5) WEIGHTED CURVATURE-GUIDED MODEL
In this paper, the following adaptive weighted model with curvature term and divergence-free vector constraints is presented where λ is a positive parameter, q is the divergence-free vector, c (x) is a weighted function defined by (6), h (x) defined by (7), ϕ (|∇f |) is the gradient dependent curvature parameter, which will be introduced in the next section, andk c,h is the weighted level set curvature [29], which is defined as follows

6) THE COEFFICIENT OF CURVATURE
The proposed model combines the TV regularization term with the curvature term as the smoothing term. To make the optimal solution closer to a piecewise constant image that is formed by homogeneous regions separated by sharp edges, we want to find a function to balance the relationship between the TV regularization term and the curvature term. While Nikolova et al. [39] proved that it is reliable to use non-convex regularization in variational image restoration, and confirmed that non-convex regularization has a good performance in edge preservation. Since curvature happens to be a function of gradient, we use a non-convex function to modify the curvature regular term. A non-convex function we choose is where a is a user-specified parameters (a > 0), and f is the input image.

B. ITERATIVE ALGORITHM BASED ON THE AUGMENTED LAGRANGIAN METHOD
We employ the alternate direction method of multiplier (ADMM) [40] to solve the proposed new model. Alternate direction method of multiplier is an important tool for solving the constrained optimization problems with separable structures. It is widely favored in the fields of image processing [41] and machine learning [42]. By introducing three auxiliary variables ω, z, g, model (8) can be transformed into the following constraint minimization problem: According to the ADMM framework, let = {u, ω, z, q, g}, given initial b 0 1 , b 0 2 , b 0 3 , b 0 4 , the solution of (11) can be found by solving the following problem b k+1 where µ 1 , µ 2 , µ 3 , γ are positive parameters. For the minimization problem (12), given initial ω 0 , u 0 , z 0 , q 0 , g 0 , and then update ω k+1 , u k+1 , z k+1 , q k+1 , g k+1 by the following formulas For (17), the exact solution can be obtained by the soft threshold function

For (18), its Euler-Lagrange equation is
where ∇ T is the adjoint operator of ∇, and satisfies ∇ T = −div. It can be solved by fast Fourier transform (FFT).
Similarly, for q-subproblem (19), the corresponding Euler-Lagrange equation is (24) which can also be easily solved by FFT.
For z− subproblem (20), its following Euler-Lagrange equation can be obtained and solved by FFT For g− subproblem (18), it can be solved explicitly by the soft thresholding function The solving steps of proposed model (8) are as follows (Algorithm 1): (8) 1. Input: Choose the parameters λ, µ 1 , µ 2 , µ 3 , γ and the Initial values

IV. EXPERIMENTS
In this section, we will discuss the choice of the parameters and give the experimental results for the proposed algorithm.
The tested images are listed in Fig.2. Fig.2(a) is the Barbara 256 image, Fig.2(b) is the Synthetic image, Fig.2(c) is the Bishapur image, Fig.2(d) is the House image, Fig.2(e) is the Table image, Fig.2(f) is the Finger image, Fig.2(g) is the Leg image, Fig.2(h) is the Fabric image. They are rescaled in [0, 1]. The stopping criterion is to reach to meet where ε is a given threshold. The clean images in Fig.2 were used for tests in the following experiments.

A. THE CHOISE OF PARAMETERS
In the proposed Algorithm, the Gaussian convolution template 3 × 3 is used in our experiments, and the cutoff frequency of Butterworth low-pass filter is fixed as b = 40. The stop condition ε = 10 −2 and the maximum number of iterations is 200. In (6) and (7), σ = 3 and K = 1 by default unless specified. The experimental results show that the parameter t ∈ [0.001, 0.05] can obtain satisfactory decomposition results. In addition, the value of a in ϕ (∇f ) defaults to a = 0.02. The parameters in the iterative system include the trade-off parameters λ and the multiplier parameters µ 1 , µ 2 , µ 3 , γ . A smaller λ will make the cartoon smoother, while a larger one will contain some textures in the cartoon image. The experiments show that γ and λ have the same effect on the decomposition results. Therefore, both γ and λ are set to constants less than 1 in the experiments. For µ 2 and µ 3 , we find that when µ 2 is smaller, the cartoon image will become smoother; and when µ 3 is smaller, the texture face extraction will be more thorough. In Algorithm 1, the parameters are set to λ = 0.7,µ 1 = 0.3, µ 2 = 0.2, µ 3 = 0.2, γ = 0.8. The parameters of different images will be adjusted.

B. EXPERIMENTAL RESULTS
In this section, in order to prove the effectiveness of the proposed model, we compared some state-of-the-art image   [7]; (b) cartoon and texture images using [9]; (c) cartoon and texture images using [28]; (d) cartoon and texture images using [14]; (e) cartoon and texture images using [29]; (f) cartoon using (8).  (b) cartoon and texture images using [10]; (c) cartoon and texture images using [28]; (d) cartoon using (8).
decomposition models [7], [9], [10], [14] [28], [29] in the following experiments. Those parameters in these models are manually mediated to achieve good results. In addition, for the proposed model (8), when take the value of the special case, i.e., let c (x) , h (x) , and ϕ (|∇f |) all equal to 1, we can get the solution of model (5), then u 1 and k 1 can be obtained by solving model (5) and used for the calculation of model (8).
Since the divergence-free vector field q has an impact on the vector modulus and direction of the texture, the texture v is equal to the divergence of g, so the vector field of g is shown in the experiments. Fig.3 shows finger image decomposition FIGURE 6. Bishapur image decomposition experiment. (a) cartoon and texture images using [7]; (b) cartoon and texture images using [10]; (c) cartoon and texture images using [28]; (d) cartoon and texture images using [14]; (e) cartoon and texture images using [29]; (f) cartoon and texture images using (8).

FIGURE 7.
Barbara 256 image decomposition experiment. From left to right: (a) cartoon, texture and texture g vector field using [7]; (b) cartoon, texture and texture g vector field using [10]; (c) cartoon, texture and texture g vector field using the proposed model (8); (d) the local texture g vector field of [7]; (e) the local texture g vector field of [10]; (f) the local g texture vector field of (8).
of OSV model and the proposed model (5). As illustrated in Fig.3(a), we can find out that our model is capable of edgepreserving, this is mainly because the curvature termk in (5) FIGURE 8. Synthetic image decomposition experiment. (a) cartoon, texture and texture g vector field using [9]; (b) cartoon, texture and texture g vector field using [10]; (c) cartoon, texture and texture g vector field using (8).
can deal with the image corner better than general TV regularization. Therefore, it is essential to include the curvature term to preserve the edges and the contour of cartoon. In addition, the g vector field of our model is superior to the OSV model. Fig.4 conducts a study about house image to show the effectiveness of curvature term and its coefficient. As shown in Fig.4(a) and (b), we can see that some textures incorrectly remain in the cartoon part. Besides, the window edges and roof of the house in Fig.4(c) and (d) are obviously excessively smooth. The regular terms of the models in [29] and our FIGURE 9. Leg image decomposition experiment. (a) cartoon, texture and g vector field using [7]; (b) cartoon, texture and g vector field using [9]; (c) cartoon, texture and g vector field using [10]; (d) cartoon, texture and g vector field using (8).
model contain curvature terms, so the edge preserving ability of the cartoon decomposed by these two models is obviously better than the compared methods. However, compared with the proposed model (8), we can see from Fig.4(e) that the top and edge of the house chimney decomposed by [29] are too smooth, and our cartoon is clearer and more similar to the main geometric features of original image.
In Fig.5, we use table image to compare various methods for which we try to show that our method is capable of preserving edges. As we can see in Fig.5(a) and (b), the structural edges are severely smoothed, especially at the edges of table legs and arm. At the same time, rsuch as the tablecloth in the upper left corner. Although the results of Fig.5(c) and (d) have no significant difference in texture extraction, the proposed model seems to have better capabilities to make the edge sharp. Compared with other methods, the cartoon of our model retains more image features, and the edges of the table legs and arm are well maintained. Therefore, our curvature-guided model outperforms a number of cartoon-texture decomposition techniques. Fig.6 displays the decomposition results based on the bishapur image. For this image, we take the value of K in c (x) as 20. From the decomposed cartoon images, we can see (a) cartoon, texture and g vector field using [7]; (b) cartoon, texture and g vector field using [9]; (c) cartoon, texture and g vector field using [10]; (d) cartoon, texture and g vector field using (8).
that Fig.6(b) contains some textures which are not extracted in the background area; The background of Fig.6(a) and (c) has block effect and the top edge of the harp is blurred; and there are some obvious grey patches on the upper left area of Fig.6(d). Therefore, our model has achieved better decomposition result than these comparison methods. Because the data fitting item of [29] has Gaussian smoothing operator intended to suppress image intensity oscillations, their cartoon is cleaner than ours. Fig.7 shows Barbara 256 image decomposition results of several experiments. Only the models which use the H −1 functional have divergence-free vector field, so only those models are compared in Fig.7. The cartoons in Fig.7(a) and (b) are blurred and the texture extraction is not complete. In contrast, the cartoon image obtained by (8) can preserve sharp edges and the texture is well removed. It can also be observed from the g vector fields that the models of [7] and [10] have many chaotic vectors in the background area, while the g vector field of the new model is cleaner. The main reason for this phenomenon is that the divergence-free vector q makes the information of g vector field more accurate. Besides, we can see from the collar of the images that the structural edges of the proposed model (8) are much sharper than those of the competitors.
In Fig.8, we select synthetic image as the test image, and the parameters are set as λ = 0.7, µ 1 = 10, µ 2 = 1, µ 3 = 2, γ = 0.8. It can be clearly found from Fig. 8(b) that some textures of repetitive patterns incorrectly remain in the structural part, and there are many artifacts at the edges of the triangle in the image. Obviously, the proposed model exhibits superior performance in edge and detail preservation. In addition, for Fig.8(c), the elliptic part of the g vector field of our model has neat lines, and the edge of the triangle is clear. Fig.9 and Fig.10 are some comparative experiments, and the g vector fields corresponding to each model are shown. For OSV model and [10] in Fig.9, the cartoon is blurry and the edge of pants contains obvious structural information in texture. Besides, the g vector fields of the models for comparison are very messy. In contrast, the cartoon of our model can make the edges sharp and the g vector field is much ordered. At the white mesh of the fabric image in Fig.10, the g vector fields of other contrast models are almost indistinguishable, and the cartoon does not have a good edge protection ability at the corner of each rhombus. It can be seen that our method provides competitive and superior results on the test image.

V. CONCLUSION
In this paper, we propose a curvature-guided cartoon and texture decomposition model with divergence-free vector constraint, which is obtained by analyzing the features of different image regions and the theory of divergence-free vector field. Some measures are proposed to achieve a good decomposition results. Firstly, the curvature term is used to filter the high-frequency components of the image to achieve better smoothing, and the edge indication function is used for weighted regularization to maintain the image edge; Secondly, in order to balance the relationship between weighted regularity and curvature term, we design a function about gradient as the parameter of curvature term, which is beneficial to make the edge sharp; Finally, we preserve the divergence-free vector field neglected when using the H −1 functional to measure oscillation function, and show the vector field corresponding to the texture in the experiments. The experimental results demonstrate that the proposed method is superior to the state-of-the-art methods.