Active contour model for image segmentation with dilated convolution filter

ACMs have been demonstrated to be highly suitable as image segmentation models for computer vision tasks. Among other ACM, the local region-based models show better performance because they extract the local information regarding intensity in the neighborhood and embed it into the energy minimization function to guide the active contour to the boundary of the desired object. However, the online segmentation of noisy and inhomogeneous is still a challenging task for local region-based ACM models. To overcome this challenge, This paper proposes a novel region-based active contour model, named active contour model with local dilated convolution filter (ACLD). The ACLD integrates local image information in the form of a signed pressure force function. Then, a Gaussian kernel is applied using dilated convolution instead of discrete convolution for regularizing the level set formulation. Finally, instead of using a constant stopping condition, the ACLD automatically stops at the object boundaries. The proposed model shows improved image segmentation results visually combined with less computational time in the case of synthetic and natural images compared with the state-of-the-art models. Further, on the ISIC2017 dataset, the ACLD yields segmentation results with the highest accuracy.


I. INTRODUCTION
W ITH the advancement of the field of computer vision, image segmentation is becoming more important. Image segmentation methods are classified into two major types [1], namely, discontinuity-based and similaritybased methods. The popular similarity-based models are deformable methods, which are also called active contour models (ACMs). The underlying concept of the ACMs is to work on the initial curve and evolve it into the desired object by using different types of formula functions to minimize the energy. Among the ACMs, level set methods [2]- [7] have been widely studied and tested for image segmentation. Level set methods are classified into the following two types: edgebased ACMs [8]- [11] and region-based ACMs [12]- [16]. The Chan and Vese (CV) model [17] that is derived from the study in [18] is the earliest and most prominent ACM.
In [2] the region fitting energy ACM was introduced for image segmentation. However, this model has limited use due to its high computational cost and the sensitivity of initial contour. LIC-ACM [3] level-set proposed the locally weighted intensity clustering property. LIC-ACM use the K-means clustering for the locally weighted intensities instead of clustering variance, which leads to substandard segmentation results. The local binary fitting (LBF) energy [19] and other ACMs were proposed in [20], [21] to solve the problem of inhomogeneous image segmentation. These models directly estimate the geometric characteristics of the evolution curve of the active contour, and they satisfactorily address the inhomogeneous image segmentation problems. Nevertheless, these methods are sensitive to the initial position of the contours; therefore, their use in practical applications of segmentation is questionable. By extending the LBF presented in [19], Zang and Song proposed [22] the active contour driven by local image fitting (LIF) energy, as well as another method of Selective Binary and Gaussian Filtering regularization level set (SBGFRLS) [12]. LSACM [4] was also proposed for specifically intensity inhomogeneity problems in medical imaging segmentation. LSACM handles the intensity inhomogeneity by mapping the input image into higher dimensions of Gaussian distribution by sliding window. LSACM uses the maximum likelihood method for segmenting the local regions since LSACM cannot perform online segmentation due to high computational costs. Weighted hybrid SPF (WHRSPF) [6] ACMs proposed for unclear object boundaries. WHRSPF is the hybrid model, which considers both local and global information intensities. Due to consideration of global intensity information, WHRSPF performance is questionable with inhomogeneous images. In Hessian matrix-based (HM-ACM) [5], a new formulation for the LSF is proposed, which improves the segmentation performance by transforming the eigenvalue information of the Hessian matrix into an LSF. The models, as mentioned earlier, assume that the intensities of the image are uniform, and they use the fixed force obtained from the SBGFRLS; this is not reliable because there are various types of images in real-world scenarios. Thus, we cannot use the same force on images with different intensities that are not uniform. Previous studies and experiments have shown the significance of Gaussian distribution and kernel in segmenting the inhomogeneous intensity images. To formulate the signed pressure force (SPF), the SBGFRLS model uses local statistical information of image intensities from inside and outside of the contour, and this SPF is responsible for the direction toward which the contour should move. Then, the Gaussian filter is used to regularize the ACM. The online region-based ACM (ORACM) [23] redeveloped the SBGFRLS [12] and introduced an ACM based on online regions. The ORACM is based on a user-defined initial contour and level set function (LSF) instead of the gradient of the LSF, which gets updated after each iteration. This method uses morphological operations (opening and closing) to smooth the LSF instead of Gaussian filters, such as SBGFRLS and LIF models.
Liu [7] proposed a new ACM (GLSEPF) with penalty energy using distance-regulated LSF for smoothness of the contour and its evolution process around the object boundary. In GLSEPF, the distance feature of the contour is constrained to 1 in the evolution process. The multi-scale level set method in [24] also shows the importance of Gaussian kernel filtering through using a Gaussian kernel as the local maximum description difference. In [25], dilated convolution (atrous convolution) filtering is proposed; it was shown that the context module increases the efficiency of segmentation models of deep convolutional neural networks. To this end, we propose a new ACM method to enhance the segmentation results of inhomogeneous images, referred to as the active contours with local dilated convolution filter (ACLD), yielding higher accuracy and less computational complexity. In particular, the significant contributions of this study are as follows: • A complex ACM is investigated by integrating local en-ergies into the LSF and regularizing it using a Gaussian kernel with dilated convolution, whereas discrete convolution is used in the regularization step of traditional active contours. • We propose a new LSF that uses an SPF function that is formulated using local intensity fitting energy. • Previous models are sensitive to initialization of contour and biased. In contrast, the ACLD is insensitive to the initial position of contour owing to robust SP F ACDL function given in (22). • The proposed model uses dilated convolution with a dilation factor of 2. As shown in Figure 5, dilated convolution has a more receptive field (5x5) without loss of resolution of the input image while using the same computation time as discrete convolution. • Finally, for qualitative and quantitative analysis, the ACM that is extensively tested on both synthetic and natural images [26] and the ISIC2018 dataset [27], [28].
The experimental results of the ACLD model confirm the efficiency and superiority over the state-of-the-art.
The rest of this paper is organized as follows. The related research is briefly described in order to better understand and develop the ACM in Section II. In Section III, we provide the formulation and a detailed description of the proposed ACM. Then, in Section IV, a comparison with well-known ACMs and the experimental results are presented. Finally, in Section V, the conclusions and discussion of future research are provided.

II. RELATED WORK
Numerous active contour methods are widely used in different computer vision-based applications for segmentation tasks. For example, the CV [29] proposed an ACM for segmentation that works well on synthetic uniform images. Still, it does not work well on complex natural images and has a higher computational cost. Another model, i.e., SBGFRLS [12], introduced the SPF and regularizing term with Gaussian filtering to eliminate the area and length parameters from the CV's energy functional of the active contour owing to its high computational time. However, although the area and length parameters are removed, the SGBFRLS does not work well as it has a higher computational time as discrete convolution.

A. SBGFRLS
The SBGFRLS [12] proposes the SPF ACM, where the user initializes the active contour. Let I : Ω → ℜ an input image and evolving curve is ℑ within the image domain Ω. As the boundary: ω ⊂ Ω and ℑ = ∂ω. In short, the ℑ inside represents the region ω and ℑ outside represents the region Ω\ω. This initial active contour continuously evolves at every step of the iteration with a SPF. SPF that is defined as follows: where ℑ i and ℑ o are the constants defined in Eqs (2) and (3), respectively. Zhang [12] also used the α and Heaviside function with eps = 0 instead of the simple CV energy minimization function. This SP F SBGF RLS function is responsible for the movement of the active contour. If the contour is outside of the object, it shrinks, and if the active contour is lying inside the object, it expands.
ℑ i and ℑ o are described as: Furthermore, they proposed the new variational level set formulation method, defined as: where α is constant and it is responsible for the speed of the level set update. where H(ϕ) represents the Heaviside function and δ(ϕ) is the Dirac function. The graphical representation of the Heaviside and Dirac function is illustrated in Figure 2, and which are described as follows: The major drawback of the SBGFRLS is that it relies on α for the update of the level set and the movement of the active contour. Thus, the user must adjust the α value for different images to get a better result. Moreover, this approach cannot self-tune the constant value α. Thus, this approach is not practical owing to the wide variety of real-life images. On extent, this method uses the |∇ϕ| from [30] defined as: It takes a significant amount of time to calculate the energy minimization function for the next iteration of the current contour. Therefore, to smoothen the transition from the previous contour to the current contour, a Gaussian kernel is used with a kernel size of 5. For re-initialization of this current ϕ, SBGFRLS use the following function:

B. LOCAL IMAGE FITTING ENERGY
In the LIF model [22], the image information is analyzed by differentiating the given original image and the local fitting image. For regularization of the LSF, the LIF model uses the Gaussian kernel. This approach has two significant advantages. First, it handles the smoothness of the LSF, and second, it prevents the initialization of the level-set function VOLUME 4, 2016 on each iteration. Thus, the model performs efficiently with lower computation complexity and similar accuracy to its predecessor model called LBF [19]. The LSF is defined as: where the I LIF () is the local fitted image. The formulation for this local fitted image is defined as: where H(ϕ) is the Heaviside function defined in Eq (5) and γ 1 and γ 2 are defined as follows: where G σ is the truncated Gaussian filter with a standard deviation σ and radius of the kernel k. The Gaussian kernel is used with kernel size of 5 to smooth the current active contour.

C. ORACM
The ORACM is an online region-based ACM for segmentation. This model [23] proposed a new level-set formulation that does not require different constant parameters. In contrast, it performs on a block thresholding process on each iteration of the active contour evolution. This thresholding  affords approximate rigid results as segmentation active contour boundary. For smoothness, this method uses morphological operations. The level-set formulation of ORACM is defined as follows: where H(ϕ) is the Heaviside function defined in Eq (5), and I(x, y) is the given input image for segmentation. SP F ORACM is the SPF that is defined as: The morphological operations, such as opening and closing, are used to smooth and remove the unnecessary small regions and areas in the current active contour.

D. GLSEPF
The GLSEPF [7] is the region based ACM. This model uses the both global signed energy and local signed energy based pressure force. First, It computes the global signed energy pressure force (GSEPF) by using the energy difference between the outer and inner energies. Secondly, It computes the local pixel-by-pixel signed energy based pressure force (LSEPF). At the end GLSEPF ACM combine the both GSEPF and LSEPF by automatically balancing the weights of both pressure forces. For regularization of the GLSEPF uses the regularization and penalty function to avoid the contour re-initialization problem.
The global SEPF function is defined as: where E g SEP F (I) can be redefined in term of motion of the ACM as: where c 1 and c 2 are the inside and outside average intensities of the contour, which is defined as: the local SEPF can is defined as: Gaussian kernel is used for the local information extraction, the local information extraction function is defined as: where m 1 and m 2 are the average intensities of the inner and outer regions of the evolution contour. which is defined as: The final evolution curve for the GLSEPF can be defined as: where w l and w g are the weighted variable for the local and global variance in the input image. µ and v are the constants which helps the balancing the energy force between global and local forces. Therefore, this model have high computational cost due gradient and calculating the both local and global energies with penalty function. In online segmentation this computational cost is really effect the results.

III. PROPOSED MODEL
In this section, we present an ACM for robust image segmentation. The proposed model consists of three major parts. Firstly, a SPF function inspired by LIF [22] is formulated using the local image information. Next, the Gaussian kernel with dilated convolution that is inspired by the multi-scale level set method [24], [25] regularizes the SPF function. The dilated convolution also shows improvement in the results as compared with the discrete convolution. Secondly, the energy minimization is done by the newly proposed level-set formulation with the help of Heaviside function and Dirac function. Finally, the automatic stopping conditions check for the active contour at the object boundary. A flowchart of the complete process of the proposed ACLD model for a given input image that is segmented into a binary image, is illustrated in Figure 1. We divided the proposed model for this method into the following subsections.

A. FORMULATION OF SPF FUNCTION
The SPF function is the critical part and backbone of the ACMs, which is responsible of the evolution process of the contour from its current position towards the object boundary. Based on observation and different experiments we are able to construct the modified SP F function which is follows: where ψ 1 and ψ 2 are the constants which are defined as follows: where G σ k is the Gaussian kernel with Dilated convolution, with a standard deviation value of 0.5, and the size of a given image automatically computes kernel the size. Previous research has shown that Gaussian filtering [24] helps to highlight the local regions centered at current pixel and provides the optimal scale of local region for every pixel. The Gaussian filter is defined as: where x is the current pixel and y is the neighboring pixel. The adjacent parameter is controlled by σ k = 2k + 1. Figure  4 shows the Gaussian filter's kernels with different standard deviations (σ).

1) Dilated Convolution
The dilated convolution has been known as "convolution with a dilated filter". It is also used in "algorithme a trous". Let J : P 2 → R be a discrete mathematical function, Ω a = [−a, a] 2 ∩ P 2 and κ : Ω a → R be a discrete kernel with the size of (2a + 1) 2 . The operator for discrete convolution is * and is defined as follows: For a general operator, let ι be a dilation factor, where * ι is defined as follows: where * ι refers to a dilated convolution or an l-dilated convolution. The visual representation of dilated convolution can be seen in Figure 5. This kernel is responsible for the smoothness and prevents the re-initialization of the SPF function at every iteration during the contour evolution. Unlike SGBRLF, LIF, and ORACM models, the proposed method is not dependent on the user-defined initial contour. Instead, it automatically detects the image size, creates the initial contour, and calculates the signed pressure force from the initial active contour.

B. LEVEL SET FORMULATION
The final LSF is defined as: where ϕ(x, y) is the current active contour at the n th iteration. The significance of the proposed level-set method and the reason for the selection of this level-set method is described as follows: • It is a parameter-free method that does not require parameter initialization, unlike previously formulated ACM. • It also does need to calculate the curvature approximation of the ∇ϕ, which tends to take a lot more computational time that makes the active contour movement towards the desired object very slow.

C. ALGORITHM
The procedural steps of the proposed ACMs are summarized in Algorithm 1: Algorithm 1 Proposed Model 1: Initialization of initial contour (ϕ) according to the size of given image I(x, y).
where p is a constant parameter, Ω 0 is a subset of the image domain Ω, and ∂Ω 0 is the boundary of Ω 0 . 2: Apply the Gaussian filter with dilated convolution on the given image I(x, y) 3: Calculate ψ 1 and ψ 2 using the Eq (13) 4: Calculate the SPF function using the proposed SP F ACLD function (21) 5: Minimize the energy using the energy minimization function using Eq (26) 6: Algorithm ends, if the termination condition is met, otherwise return to step 3. 7: Apply Gaussian filter with dilated convolution on SP F ACLD function Eq (21) for smoothing the final contour curve.

D. TERMINATION CRITERION FOR ACTIVE CONTOUR EVOLUTION
The curve evolution using energy minimization extends the active contour at the boundaries of the desired object. So, it is an integral part of the curve evolution that automatically stops at the object's boundaries. Numerous ACM takes the parameter, such as iteration number, to evolve an active contour for a certain evolution step. However, the proposed model uses an automatic checkpoint to stop the evolution process of the active contour curve. This automation step depends on the area of L 1 term. As in the Figure 3 shows the positive and the negative energy (as long as the contour's energy is greater than 0 or less than 0); it implies that the contour is not at the object's boundary and that it has to evolve to reach the object's boundary where the energy will be zero.

IV. SIMULATION AND EXPERIMENTAL RESULTS
This section explains the different types of experiments performed to evaluate the proposed model's results using synthetic, natural, and medical images. We also compared the proposed ACLD model with other well-known ACMs, such as SBGRLFS [12], LIF [22], ORACM [23] and GLSEPF [7], LSACM [4]. The proposed ACLD model was implemented in MATLAB 2020b on a 2.8 GHz Intel Core processor with 16GB of RAM.

A. DATASETS
Various measures are available to evaluate the segmentation results of different models. We used three significant types of images for our model's evaluation. The first group was computationally generated synthetic images. These types of   [27], [28].

B. EVALUATION MEASURES
For quantitative measurements and evaluation of the segmentation results of the proposed model as compared with other state-of-the-art models, such as LIF [22], GLSEPF [7], and LSACM [4]. We used BF (Boundary F1) Score [31] and Hausdorff distance [32] for segmentation performance (Presented in Eqs (28)-(29)).
BF score(I(x, y), P (x, y)) = 2 * p * r (r + p) Where I(x, y) is the input image and P (x, y) is the binary mask of segmentation result, r is the Recall, and p represents the Precision. BF Score is the contour matching score used for the evaluation of the image segmentation techniques. In our example, the two groups are the object's binary mask, and the segmentation results from the active contour. Hausdorff distance is broadly used as a performance measure to calculate the distance between two points. The applications of Hausdorff distance in medical image segmentation are significant. Hausdorff distance can be defined as: Where X and Y are the two given points, Hausdorff distance is used to calculate the difference between the binary mask of the given image and the segmentation mask generated by the ACMs.

C. EXPERIMENTAL RESULTS USING SYNTHETIC IMAGES
For evaluating the proposed ACM, we tested it on different types of images for segmentation. Firstly, we applied our model to synthetic images with simplistic structures and uniform intensity distribution within the regions of interest.   These artificial images are computer-generated grayscale images usually used to evaluate different segmentation models. Figure 6 shows qualitative segmentation results and comparisons with the state-of-the-art methods using different synthetic images. From the first column to the seventh column it shows, the original grayscale images with initial active contour, segmentation results of the LIF energy model, segmentation results of the SBGFRLS model, segmentation results of the ORACM model, segmentation results of the GLSEPF model, segmentation results of the LSACM model and the proposed ACLD model, respectively. The LIF model failed to detect the boundaries properly. This is the reason for not distinguishing between the regions of interest and background; thus, this resulted in region overlap in the segmentation result for all images. In contrast, the SBGFRLS model could properly segment first and the third images but could not correctly segment the second one owing to intensity inhomogeneity in the regions of interest. As shown in the results, the contour could not precisely follow the boundaries at the front part of the wings of the airplanes. The ORACM model partially segmented the first image, where a small portion within the black region could not be segmented. In contrast, it could not correctly segment the airplanes in the second image and could not fully detect the boundaries of both regions in the third image. GLSEPF model is able to segment all the objects from images with high computational cost and number of iterations. While, LSACM perform very poorly on all the images and unable to segment the objects. In Image 2 the LSACM is totally unable to detect even one plan among three. In Image 3 LSACM is only able to detect the shapes however is unable to differentiate between two objects. Mostly, ACM's like SBGFRLS, GLSEPF, LIF and LSACM uses the gradient information α of the given image to control the speed and evolution of the current contour. This is significantly computation cost for the models to segment the given images. In contrast, the proposed ACLD could  correctly segment the regions of interest in all images, which shows that the ACLD outperformed the state-of-the-art models qualitatively. It segmented a few tiny speckles of noise in the black regions of interest in noisy image 1, which is ignorable. While, ACLD does not use the gradient information α like ORACM that's why ACLD take less computational time and less iteration. Table 1 show a comparison among the state-of-the-art active contours and proposed ACLD method in terms of the execution time in seconds and the number of iterations taken by models to reach the final results, as shown in Figure 6. It shows that the proposed ACLD model outperformed the state-of-the-art active contours, with minimum execution times of 0.0018, 0.0012, 0.005s and number of iterations of 4, 3, and 8 for Image 1, Image 2, and Image 3, respectively, from Figure 6.

D. EXPERIMENTAL RESULTS USING REAL IMAGES
This section describes the effectiveness of the proposed ACLD ACM. To verify the validity of our model, we used the real images from the real-world Weizmann dataset (Seg-VOLUME 4, 2016  mentation evaluation database) [26]. This dataset is consists of different types of complex objects such as airplanes, and buildings. Figure 7 shows qualitative segmentation results and comparisons with the state-of-the-art methods using different real images. From the first column to the eighth column it shows, the original grayscale images with initial active contour, segmentation results of the LIF energy model, segmentation results of the SBGFRLS model, segmentation results of the ORACM model, segmentation results of the GLSEPF model, segmentation results of the LSACM model and the proposed ACLD model, and human-annotated segmentation of objects respectively. Some ACM models use the gradient information α to estimate the directional movement of the contour. This is significantly expensive cost in terms of computational time. Therefore, ACM's (LIF, SGBFRLS, GLSEPF and LSACM) shows very large amount of compu-tational time, which is unacceptable in online segmentation.
And also, Table 2 shows the iterations and time takes each active contour model in Figure 7 to compare the quantitative analysis of the different active contour models. Figure 7 and Table 2 clearly show that our proposed ACLD model outperforms all the other state-of-the-art active contour models in terms of execution time and number of iterations.

E. EXPERIMENTAL RESULTS USING MEDICAL IMAGES
For further experiments, we testes the proposed ACLD model on ISIC 2018: Skin Lesion Analysis Towards Melanoma Detection [27], [28] and also compared ACLD model with other state-of-the-art models. Skin lesion is an extremely difficult task because of the noise, the presence of hair in the background and high quality images. Figure 8 shows the selective qualitative results using the ISIC2018 cancer dataset, the green contour is original mask, red contour is the ACM models segmentation results, and blue shows the initial contour. As Figure 8 shows, the LIF model performance is really noisy and it gets stuck in local minima. The SPF of LSACM and GLSEPF are unable to perform on big size images as a result the performance of LSACM and GLSEPF model are unacceptable on ISIC 2018 dataset. Contrastingly, even with presence of hairs in Figure 8, Image 1, and Image 2, the proposed ACLD model segments the cancer lesion area effectively. Further, Table 3 offers the quantitative analysis for the proposed ACLD model with LIF, LSACM, and GLSEPF models using evaluation measures. The proposed model yielded an average BF score of 96.7% and also shows the minimum average Hausdorff distance of 55.

F. INITIAL CONTOUR AND NOISE INDEPENDENCE
The major problem with ACM is that they are sensitive to noise and the position or shape of the initial contour. The proposed ACLD model overcame this problem. Figure 9 showed the independence of the initial active contour and robustness by offering experiments using various positions of initial active contours with different sizes. It shows that the proposed ACLD model could properly segment the regions of interest in all cases of initial contours that verify its independence from the initial position of contour and robustness. The proposed method was also tested and qualitatively compared using noisy Image 1, in which salt and pepper noise is added Figure 6. Due to lack of gradient informational parameters and functions, ACLD shows the robustness in segmenting the object from the given image. Although the proposed method also segmented some tiny speckles in the black region of interest, it did not yield a perfect segmentation result. However, it outperformed the compared state-ofthe-art in terms of execution time and number of iterations.

V. CONCLUSION
The state-of-the-art ACMs face challenges in segmenting images under biased conditions or intensity inhomogeneity.
To target this problem, we developed a novel ACM called ACLD for image segmentation. The energy functional of the proposed ACLD model is devised using a SPF function that is formulated using local image energies. For regularization and to update the current contour, we used dilated convolutions with the dilated factor of 2. The proposed ACLD model is tested on diverse images, including synthetic, natural [26], and medical images from a skin challenge [27], [28]. The proposed ACLD model outperformed the state-of-theart models both qualitatively and quantitatively. Moreover, it also yielded lower execution time and required less iteration than the state-of-the-art models.