Three Adaptive Sub-Histograms Equalization Algorithm for Maritime Image Enhancement

According to maritime image histograms’ statistic and analysis, the histogram of pure maritime image obeys Gaussian distribution approximately, thus Three Adaptive Sub-histograms Equalization (TASHE) algorithm for maritime image enhancement is proposed in this paper. First, the characteristics of pure maritime image’s histogram are studied, then the adaptive threshold’s optimal selection strategy for the histogram’s division is discussed, finally the implement of three sub-histograms is described. This paper employs visible gray maritime image, visible color maritime image and infrared maritime image to verify the enhancement algorithm’s effectiveness and robustness, the experimental results show that TASHE algorithm can not only keep the maritime image’s mean brightness and naturalness, but also improve the maritime image’s contrast without the noise and artifacts. The objective image quality assessment also indicates that TASHE algorithm can improve the original maritime image’s Enhancement Measure by Entropy (EME) value, furthermore, when a maritime image is pre-processed by TASHE algorithm, the maritime target’s Detection Rate (DR) can be improved.


I. INTRODUCTION
Histogram Equalization (HE) [1] is one of the most widely used methods in image enhancement [2]- [5], which has the advantages of easy computation and implementation [6]. Nevertheless, the result after histogram equalization brings about some disadvantages [7] such as mean gray value's fixing, entropy's declination, and the details' missing. Aiming at the disadvantages of HE, some representative improved methods are proposed.
Some researchers study the sub-histogram equalization methods, which can enhance the image and prevent the over-enhancement, for example, Brightness Preserving Bi-Histogram Equalization (BBHE) [8] first divides the image histogram into two parts(sub-histograms), and then the two sub-histograms are independently equalized, which can preserve the original image's mean brightness. Equal Area The associate editor coordinating the review of this manuscript and approving it for publication was You Yang . Dualistic Sub-Image Histogram Equalization (DSIHE) [9] divides the histogram into two sub-histograms based on its original probability density function, which can obtain the maximum entropy. Minimum Mean-Brightness Error Bi-Histogram Equalization (MMBEBHE) [10] is the extension of BBHE, which provides maximal brightness preservation. In Jan. 2020, Khan et al. [7] proposed the method of fuzzy bi-histogram equalization, the key technique of this method is that the histogram's dividing threshold can be derived from the skewness of the histogram, which indicates that the threshold of sub-histogram equalization can be selected by the characteristic of the image's histogram. Some researchers have also studied the modified histogram equalization methods named by weighted histogram technique, thresholded histogram technique, or clipped histogram technique, finally histogram equalization operates the new histogram, the enhancement result usually has a better performance, this is because the pixel's frequency of histogram can control the image's enhancement rate. For example, Weighted Thresholded Histogram Equalization (WTHE) [11] limits and modifies the original image's histogram by a transformation function, the enhancement result can reduce the over-enhancement. Bi-Histogram Equalization with a Plateau Limit (BHEPL) [12] controls the enhancement's degree by limiting the gray value's frequency, the enhancement result can keep the original image's brightness. In the reformed process of HE technique, image's characteristics can also be used as the dividing threshold, for example, exposure based Sub Image Histogram Equalization (ESIHE) [13] combines the idea of BBHE and BHEPL, which takes the image's exposure as the threshold to divide the histogram into two sub-histograms, then the two sub-histograms are clipped by the other threshold, finally the two novel sub-histograms are equalized respectively, the ESIHE algorithm is suitable for enhancing the low exposure gray image. The Recursively Separated and Weighted Histogram Equalization (RSWHE) [14] divides the entire histogram into the sub-histogram recursively, and the High-speed Quantilebased Histogram Equalization (HSQHE) [15] divides the entire histogram by quantile values, both of two algorithm may be not effective when enhancing the maritime image, because the principle of two algorithms doesn't consider the characteristics of the histogram of maritime image. To enhance the image's local details, some researchers improved the idea of HE in the histogram of an image's local area. For example, Partially Overlapped Sub-block Histogram Equalization (POSHE) [16] and Local Histogram Equalization (LHE) [17] use a small window that slides through every image pixel sequentially, and only pixels within the current position of the window are histogram equalized, both of the two algorithms take local information into consideration when enhancing the image, which can enhance the image's details obviously, but over-enhancement are sometimes occurred. The idea of HE can also be used in the Discrete Wavelet Domain (DWT) [18], when the original image is decomposed into low-frequency and highfrequency components by DWT, the low-frequency coefficients' histograms are equalized, the high-frequency coefficients are kept constant and the reconstructed image can be obtained, finally weighting the original image and the reconstructed image can get the better performance's result. Some researchers propose a new histogram's definition and concept which can better reflect the image's variation information, for example, Singh et al. [19] considered that the rough region of the image can reflect the image's feature information more effectively, so a novel histogram can be derived from the image's rough region, then the entire image's mapping function can be calculated by the novel histogram equalization, in his work, two algorithms named by Dominant Orientation-based Texture Histogram Equalization (DOTHE) and Edge-based Texture Histogram Equalization (ETHE) are proposed. However, if a maritime image appears some small targets, in such case, the small target's region cannot reflect the image's feature information effectively, so the enhancement result often has some artifacts when DOTHE and ETHE algorithms are used to enhance the maritime image. In the study of image enhancement of sea water, Song et al. [20] established a variational model for enhancing the maritime image without any maritime target, but as for a maritime image including a maritime target, Song's algorithm may not be enough effective and has less help in the maritime target's tracking and recognition. In our work, we consider the characteristics of maritime image's histogram, optimize and improve the idea of the sub-histogram equalization, the design of our algorithm is equalizing the three sub-histograms by two adaptive thresholds to enhance the maritime image, and then TASHE algorithm is proposed in this paper.
The remainder of this paper is organized as follows: Section II introduces the characteristics of the maritime image's histogram, Section III discusses the adaptive threshold's selection derived from the original image's histogram and the optimal strategy of threshold selection, Section IV describes the implement of TASHE algorithm, Section V shows the experimental results and assesses the enhancement result's objective quality of each algorithm, and Section VI concludes the paper.

II. RESEARCH ON THE CHARACTERISTICS OF MARITIME IMAGE'S HISTOGRAM A. RESEARCH ON THE CURVE'S VARIATION OF MARITIME IMAGE'S HISTOGRAM
In order to explain the characteristics of maritime image's histogram and make some investigations and demonstrations sufficiently, four visible gray maritime images, four visible color maritime images and four infrared maritime images are shown as an illustration in Fig.1, and all of the maritime images contain the pure sea background and include some small maritime targets such as boat, yacht and surfer.
Mean filter with the size of 1 × 13 is used to denoise the original histogram in order to make the histogram's curve smooth, which can be observed more clearly, the filtered histograms of the above-mentioned maritime images are shown in Fig.2, as for Fig.2(b), we show the V component's histogram in HSV color space, from which, it can be summarized that the histogram curve of each maritime image rises firstly and then arrives at a peak value, finally the curve declines near to zero. According to the curves' variation tendency, it can also be seen that the histogram of a maritime image obeys the Gaussian distribution nearly.
On the study of image histogram's fitting, Celik et al [21]. use the Gaussian Mixture Model (GMM) to model the image's histogram, if an image can be decomposed into the background region and target region, two-order GMM can fit the image's histogram in the common case. According to Celik's work [21], as for a maritime image with pure sea background, the sea background is large, but the maritime target's region is small, which can be ignored, thus one-order Gaussian distribution can fit a maritime image's histogram instead of GMM. In the following, Gaussian distribution is used to fit the maritime image's histogram with estimated parameters by Maximum Likelihood Estimate (MLE) law [22].

B. PARAMETER ESTIMATION AND SIMILARITY ASSESSMENT
Now, we use Fig.3(a) (enlarged by Fig.1(b)) as a illustration to explain how to use Gaussian distribution to fit a maritime image's histogram, the experimental maritime image's actual histogram and Gaussian distribution's fitting curve are shown in Fig.3(b). According to MLE law,μ andσ in Gaussian distribution can be estimated respectively according to (1).
In the above formula, N is the total number of the image's pixels, x denotes the mean gray value of the image, and x i denotes the ith pixel's gray value in the image.
According to (1),μ = 132 andσ = 11 are calculated as two estimated parameters of Gaussian distribution for fitting the Fig.3(a)'s actual histogram.
In order to better explain that maritime image's histogram obeys Gaussian distribution approximately, the PSNR (Peak Signal Noise Ratio) between the actual histogram and the fitting Gaussian distribution is proposed in (2), where MSE denotes the Mean Square Error between h(i) andĥ(i), h(i) denotes the actual histogram's pixel frequency, andĥ(i) represents the fitting distribution's pixel frequency, the maximum value of (h(i) −ĥ(i)) 2 is equal to 1, which is put as the numerator of the mathematic expression, the larger PSNR value has, the better the fitting Gaussian distribution approaches to the true histogram.
From Tab. 1, each histogram's PSNR value is more than 40dB, which can reflect that the actual maritime image's histogram is quite similar with the fitting Gaussian distribution.

III. THE DISCUSSION OF OPTIMAL THRESHOLD SELECTION'S STRATEGY
As we mentioned above, the middle gray-value range occupies the dominated part in the total histogram, thus the total of the frequency in the middle range should be preserved in the process of sub-histogram equalization in order to keep the original image's mean brightness, otherwise it can cause some distortion and artifacts, such as another algorithm's result in the Section V. Three sub-histograms equalization by two thresholds is reasonably designed for enhancing the maritime image, and in this section, the optimal threshold selection's strategy is proposed after the comparison of different enhancement performances caused by different thresholds' selection methods.

A. THE DISCUSSION OF TWO THRESHOLDS' LOCATION SELECTION
Without loss of generality, three different methods on threshold's location selection are discussed as the three following cases (t 1 and t 2 denote the two thresholds respectively): Case 1: Both of the two thresholds locate on the left side ofμ, the values of t 1 = 40 and t 2 = 80 are set as an example for illustration. Case 2: One threshold locates on the left side ofμ, and the other threshold locates on the right side ofμ, the values of t 1 = 100 and t 2 = 170 are set as an example for illustration.
Case 3: Both of the two thresholds locate on the right side ofμ, the values of t 1 = 160 and t 2 = 200 are set as an example for illustration.
We use the image's Mean Brightness (MB) to select the reasonable threshold's direction, in addition, the MB value can be denoted as (3).
where N is the total pixel numbers of an image, I (i, j) is the gray value at the ith row and the jth column of the image's matrix, H is the height of the image's matrix, and W is the width of the image's matrix. The enhancement result by three sub-histograms equalization under three above-mentioned threshold selection's method is shown in Fig.4, and the MB of the enhancement result with three different cases is shown in Tab. 2. It can be seen that the MB value of Fig.3(a) is 132, if one threshold locates on the left side ofμ, and the other threshold locates on the right side ofμ, the MB value after the enhancement result is 135, which can preserve the original image's brightness and produce less noise. In conclusion, the optimal threshold location selection's strategy is that one threshold locates on the left side ofμ, and the other threshold locates on the right side ofμ. Since Gaussian distribution is symmetrical with respect toμ, the two thresholds can be set with the same distance towardsμ, how to select the optimal length is mainly discussed in the following.

B. THE DISCUSSION OF TWO THRESHOLDS' DISTANCE SELECTION
In this section,μ is set as the center, two thresholds with different length centered byμ are discussed, since the maritime image's histogram obeys the Gaussian distribution approximately, 1σ law makes the second sub-histogram have the pixel's proportion of 68.3% approximately, 2σ law makes the second sub-histogram have the pixel's proportion of 95.2% approximately, and 3σ law makes the second sub-histogram have the pixel's proportion of 99.7% approximately, enhancement results under three different laws are shown as Fig.5.
In Fig.5(a) and Fig.5(b), the results under 1σ and 2σ law exist some noise and artifacts in the sea region, thus the performance of 3σ law is more suitable for the maritime   image enhancement. The result's histogram after three different laws is shown as Fig.6, from which, it can be seen that 1σ and 2σ law make a certain number of pixels distribute on the first sub-histogram (low gray intensity's range) or the third sub-histogram (high gray intensity's range), by contrast, 3σ law makes only less pixels distribute on the first sub-histogram and the third sub-histogram, a large number of pixels distribute on the second sub-histogram (middle gray intensity's range), thus the enhancement result can suppress the noise and artifacts.

IV. THE IMPLEMENT OF TASHE ALGORITHM
The proposed algorithm named by TASHE in this paper can be chiefly described in Tab. 3. On the adaptive thresholds selection of Step2, according to 3σ law in the statistics, the calculation method proposed for t 1 and t 2 is shown as (4), where k can be chosen under VOLUME 8, 2020   Fig.1(a-1). the range of [3,5], in this paper, we chose k = 3 for the threshold's selection. In some peer researchers' work, LCM (Local Contrast Method) [25] detects the suspected target by the saliency of local contrast, the threshold is also put by the average of local contrast under 3σ law. In the automatic infrared target segmentation's algorithm based on the structure tensor [26], the adaptive threshold for the eigenvalue is selected by the mean value and the standard deviation derived from the statistics of eigenvalue, the main principle is similar with (4).
Since t 1 and t 2 are derived from the original image's histogram, our proposed algorithm is adaptive from different image. In the following, the two thresholds divide the image's total gray range into three sub-intervals, which is shown in (5), correspondingly, three sub-images after sub-histogram equalization can be written as (6) according to the gray range of three sub-intervals.
where I denotes the original image, I(i, j) represents the gray value whose pixel is in the ith row and the jth column of the image, X 1 , X 2 and X 3 denote the first sub-image, the second sub-image, and the last sub-image respectively.
In Step3, we firstly compute each pixel's frequency of the total image and summarize the three sub-histograms' frequency, then the normalized pixel's frequency of each sub-histogram is obtained, finally we need to compute the accumulated normalized frequency of each of three subhistograms. When x is denoted as the image's gray value, obviously x has three different value's ranges according to eq.(6), when x ∈ X 1 , we accumulate the gray level's frequency in the first sub-histogram from 0 to x, the result can be written as CDF 1 (x), when x ∈ X 2 , we accumulate the gray level's frequency in the second sub-histogram from t 1 to x, the result can be denoted as CDF 2 (x), when x ∈ X 3 , we accumulate the gray level's frequency in the last sub-histogram from t 2 to x, the result can be denoted as CDF 3 (x).
In Step4, we need to compute the transformed gray value after three sub-histograms equalization according to the normalized pixel's frequency of each sub-histogram. First, we use the gray transformation function of traditional HE as a reference to get the sub-histogram equalization's function. The function [23] of gray value's transformation after conventional histogram equalization is described as (7).
where a represents the minimum of output's gray value, b represents the maximum of output's gray value, x represents the gray value of the input, CDF(x) represents the cumulative density function with respect to x.
is the global gray transformation function of conventional HE algorithm. As for three sub-histograms equalization, a, b, and CDF(x) should be replaced by the reasonable variable of the corresponding sub-interval, thus three sub-histograms equalization can be described as (8).
where y can denote the gray value transformation function in the TASHE algorithm, we can easily get the enhancement result Y by y.
In addition, Fig.7 also shows the histogram's change after TASHE algorithm, it can be indicated that TASHE algorithm equalizes and flattens the three sub-histograms in the three corresponding sub-intervals' gray range.  Fig.1(b-1).

V. EXPERIMENTAL RESULTS
In this section, visible gray maritime image, visible color maritime image, and infrared maritime image are chosen to verify the enhancement algorithm's validity. Fig.8(a) and Fig.9(a) are the visible gray maritime images, Fig.10(a) and Fig.11(a) are the visible color maritime images, and Fig.12(a) and Fig.13(a) are the infrared maritime images, which are acquired by a infrared camera in the helicopter above the sea. Meanwhile, we select seven representative modified algorithms based on HE for comparison, the compared algorithms are conventional or novel, the enhancement results are shown in Fig.8, Fig.9, Fig.10, Fig.11, Fig.12 and Fig.13. As for Fig.10(a) and Fig.11(a), we use HSV color space to produce the enhancement result, H component and S component among the HSV color space are kept constant, only V component is processed by each compared algorithm, at last, we transfer the new HSV color space to new RGB color space for displaying the algorithm's result.
The Enhancement Measure by Entropy [24] (EME) and the MB value are used to assess the image's objective quality in Tab. 4, Tab. 5 and Tab. 6. The enhancement result with higher EME value and nearer MB value to the original image has the better enhancement performance.

A. EACH ENHANCEMENT ALGORITHM'S RESULT OF VISIBLE GRAY MARITIME IMAGE
In Fig.8 and Fig.9, HE, MMBEBHE, BHEPL, LHE, DOTHE and ETHE algorithms cause obvious over-enhancement. BBHE algorithm causes some artifacts in Fig.8(c), which is suppressed in Fig.9(c), and the maritime target is enhanced effectively in Fig.9(c), so the performance of BBHE algorithm is not robust. From Tab. 4, it can be seen that the proposed algorithm has the largest EME value, and the MB value of the proposed algorithm is the nearest to the original image's among the compared algorithms, it can be concluded that our proposed algorithm can not only keep the original maritime image's brightness, but also enhance the maritime target. As for some newest algorithms of enhancement for Fig.8(a), the EME values of DOTHE and ETHE algorithm is 1.87 and 1.98 which are lower than the proposed algorithm, and the MB values of these enhancement results are 148 and 140, which have some changes from the original image's.
B. EACH ENHANCEMENT ALGORITHM'S RESULT OF VISIBLE COLOR MARITIME IMAGE From Fig.10 and Fig.11, it can be seen that HE, BBHE, BHEPL, LHE, DOTHE and ETHE algorithms cause obvious over-enhancement. MMBEBHE algorithm enhances  Fig.8 and Fig.9.
the Fig.11(a) effectively, the MB in V component in HSV space of this algorithm's result is 187, which is near to the original image's, and improves the original images to more than 2 times, but it fails to enhance the original maritime image of Fig.10(a), in a word, the performance of MMBEBHE algorithm is not robust, by contrast, the proposed algorithm is more effectively. Tab. 5 is the EME value and the MB value of V component, from Tab. 5, it can be seen that the proposed algorithm's EME value is the largest, and MB value of V component is near to the original image's. In conclusion, the proposed algorithm can also enhance the visible color maritime image effectively.

C. EACH ENHANCEMENT ALGORITHM'S RESULT OF INFRARED MARITIME IMAGE
In Fig.12 and Fig.13, all of the compared algorithms produce heavy distortion in such case, the proposed algorithm keeps the original image's mean brightness and naturalness, and enhances the image's contrast effectively. From Tab. 6, it can be seen that most of the compared algorithms cause serious changes in the original image's MB, by contrast, the proposed algorithm's MB value approaches to the original image's and the EME value is the largest. In a word, the proposed algorithm can also have the enough effectiveness in the infrared maritime image enhancement. VOLUME 8, 2020  Fig.1(c-1).

TABLE 5.
The objective image quality assessment of sub-image in Fig.10 and Fig.11.

TABLE 6.
The objective image quality assessment of sub-image in Fig.12 and Fig.13.
In the field of small infrared target's detection, LCM algorithm [25] detects the small or dim target in an infrared image by the saliency of local contrast between the local target's region and local background's region. When the maritime image's contrast is increased by enhancing algorithm with less noise and artifacts, the maritime target's Detection Rate (DR) can be increased correspondingly, and the False Alarm Rate (FAR) can be kept in the low level (the calculation for DR and FAR can be referred by [25]), thus our proposed algorithm can be considered as the pre-processing algorithm for the maritime target's detection and tracking. Moreover, in the filed of infrared target's segmentation [26], [27], this proposed algorithm can also improve the segmented target area's accuracy.

VI. CONCLUSION
This paper analyzes the characteristics of maritime image's histogram, and conclude that the histograms of visible gray maritime image, visible color maritime image and infrared image obey the Gaussian distribution nearly, the PSNR between the factual histogram and the fitting curve is more than 40 dB, so Gaussian distribution can be used to fit the histogram of the maritime image. According to the characteristics of Gaussian distribution, three adaptive sub-histograms equalization is designed for maritime image enhancement, the optimal threshold's selection strategy is discussed and the implement of three sub-histograms equalization is described in this paper, the thresholds are derived from the image's histogram, thus the algorithm can be adaptive to different maritime images. From the experimental results, it can be seen that the proposed algorithm keeps the original image's mean brightness, enhances the target effectively, suppresses the sea background's over-enhancement and avoids noise effectively. The objective image quality assessment can also indicate that the proposed algorithm can improve the EME value of the original maritime image, furthermore, the proposed algorithm can also be used as the effective pre-processed algorithm for maritime target's detection and tracking.
In future, we can study the valid algorithm to enhance the sea & sky image, when the image includes sea and sky background, the histogram can be divided by two sub-Gaussian distributions, correspondingly, the effective algorithm based on reformed histogram equalization can also be studied.