Low-Light Image Enhancement via Pair of Complementary Gamma Functions by Fusion

Enhanced images by the traditional gamma correction (GC) method still have low contrast within high illuminance regions. In order to enhance the visibility in dark regions and simultaneously achieve high contrast in bright regions for low-light images, this paper proposes a novel method via a pair of complementary gamma functions (PCGF) by image fusion. We first define PCGF and then show its outstanding potential for low-light image enhancement by some preliminary experimental results. In order to release its performance and verify its effectiveness, we further design a simple enhancement method for low-light images based on it by an elaborately designed fusion strategy. Two input images for fusion are derived from the enhanced image by PCGF and that by proposed sharpening method, respectively. Experiments show that our proposed method can significantly enhance the detail and improve the contrast of low-light image. The qualitative experiment results show that the proposed method is effective and the comparative quantitative assessment shows that it outperforms other state-of-the-art methods.


I. INTRODUCTION
High quality images play an important role in object detection [1], image classification [2], and saliency detection [3] and so on. However, it is generally difficult for us to obtain high quality images because of the limitations of image acquisition technology and environment. Under the condition of low illuminance, due to the surrounding environment or lack of self-exposure, images captured by optical imaging devices are overall visually dark, blurred in detail and poor visible. Many methods, such as histogram-based methods [4]- [12], Retinex-based methods [7], [13]- [22] and fusion-based methods [23]- [26], have been proposed to enhance low-light images.
Histogram-based methods are based on the modification of histogram to enhance the contrast of images with low illumination. Histogram equalization (HE) [4] and contrast-limited adaptive histogram equalization (CLAHE) [5] are commonly used due to their effectiveness in improving image contrast and detail, but their drawbacks are over-enhancement and The associate editor coordinating the review of this manuscript and approving it for publication was Andrea F. Abate . the introduction of artifacts in color images. To overcome these defects, in [6] a contrast enhancement method using layered difference representation of 2D histograms is proposed. In [7], a hybrid histogram modification method is proposed by combining gamma correction and traditional histogram equalization. However, the enhanced images look filled with gloom darkness. In [8] a so-called contextual and variational contrast enhancement (CVC) method is proposed by a histogram mapping which emphasizes large gray-level differences. However, these methods will meet difficulties when dealing with very dark regions because of the limitation of stretching range.
Retinex-based algorithms (such as SSR [14], MSR [15], and MSRCR [16]) are effective for image processing. However, the enhanced result for low-light image is often gray-out. In order to solve this problem, researchers have made much improvement for Retinex-based methods [7], [17]- [22], [27]. These methods can make the enhancement effect more significant, but the execution time is too long to be conducive to real-time processing.
Fusion-based algorithms have been widely used for image enhancement, which blend several enhanced images by VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ different methods into a single one. In [23] and [24], two fusion-based methods are developed to improve the quality of low-light images. However, the contrast in bright regions isn't high enough. In [44], a new single-image dehazing solution is proposed based on the adaptive structure decomposition integrated multi-exposure image fusion, which can greatly enhance local details of foggy images. In [45], a novel image fusion framework based on non-subsampled shearlet transform is proposed, which can improve the fusion quality of multi-modal images. In [46], a novel multi-modality medical image fusion method is presented based on phase congruency and local Laplacian energy, which can well preserve structured information and well extract detailed information. The effect of fusion-based method is obvious, and the execution time is not long. However, it is not clear how input images affect the performance during fusion. Almost these methods still have low contrast within high illuminance regions. In order to enhance the visibility of dark regions and simultaneously achieve high contrast of bright regions for low-light images, this paper proposes a novel enhancement method by fusion. We first define a pair of complementary gamma functions (PCGF). Then, we put forward a new sharpening approach further improve the details and contrast of low-light images. Finally, we design a simple fusion method to merge the value component of the enhanced image in HSV space by PCGF and the value component of the enhanced image in HSV space by sharpening.
The main contributions of this paper can be summarized in following three aspects: 1) We define a pair of complementary gamma functions (PCGF). PCGF can achieve a delicate balance between underexposure and overexposure, so it can greatly improve the contrast of dark regions and bright regions in the image at the same time. Proposed PCGF can hopefully applied in foggy image dehazing, underwater image enhancement and saliency detection and so on. 2) We design a simply and effectively weight summation method based on the average of the value component in HSV space. Either weighting factor is directly proportional to the average value component of the corresponding mapped image derived from PCGF, so the output image can achieve a moderate brightness. 3) We design an image fusion strategy which can achieve excellent enhancement results for low-light images. One input image is derived from PCGF to achieve a satisfactory global contrast and a moderate whole luminance. While the other is derived by combining traditional unsharp masking method and histogram equalization method to further improve local details. The remainder of this paper is organized as follows.
In Section II, we deliver a brief review of traditional gamma correction. In Section III, We then describe the proposed pair of complementary gamma functions (PCGF) and show its outstanding potential by some preliminary experimental results. In Section IV, we detail our low-light image enhancement method based on PCGF by fusion. The experimental results are presented and discussed in Section V. Finally, the conclusion and prospect are given in Section VI.

II. RELATED WORK
The response of cathode ray tubes (CRTs) caused by electrostatic effects in the electron gun or many imaging devices has the following nonlinear output [33]: where x and y are the normalized input image and the normalized output image, respectively. Gamma correction (GC) is associated with the above CRT gamma (1) and it is just the inverse of CRT gamma (1) as follows [34]: where γ < 1 is a constant, usually taken as 1 2.2 or 0.4545. For RGB images, the transformation is executed for three channels separately. GC has been frequently used for image enhancement. Fig. 1 shows two original low-light images and enhanced ones by GC, from which it can be seen that enhanced images have better visual effects on the whole. However, their bright areas are also enlarged which makes them too bright and also reduces the local contrast within high illuminance regions. Obviously, CRT gamma has the opposite effect on low-light images. Ju et al. propose gamma correction prior (GCP) for dehazing which is described as [31]: In the process, the input image x is firstly inverted into 1 − x, then it is processed by GC and finally inverted back to output the processed image y. It is said that the outputs of GC and GCP from haze images are inhomogeneous and homogeneous, respectively. In fact, Dong et al. [29] propose the inverse strategy for the first time and then it is used for low-light image enhancement [26], [30].

III. PAIR OF COMPLEMENTARY GAMMA FUNCTIONS
A. PAIR OF COMPLEMENTARY GAMMA FUNCTIONS (PCGF) From Fig. 1, it is obvious that GC can improve the overall brightness of low-light images, but the bright area is also increased which makes the enhanced image too bright and perhaps still has poor visibility. Therefore, GC is often used as a preprocessing step. Fig. 2 gives some original low-light images and their two corresponding maps by CRT gamma (1) and GC (2), from which it can be seen that the CRT gamma (1) can make image edge and contour clear while the inverse GC (2) can make low-illumination regions become visible. Inspired by (3), we define a new function as follows: Mapping the right side of (4) by the traditional GC, we can derive the following function:  (5) is brighter than the original low-light images.
The following is a qualitative analysis from the perspective of function. All the below analysis is based on the premise of x ∈ (0, 1). First of all, for ε < 0, the power exponent function y = x ε satisfies that: In other words, for κ 1 < κ 2 , the power exponent function y = x κ 1 satisfies that: Hence, for γ < 1, we have for GC (2): and Therefore, we can further obtain: As a result, the mapping defined by (3) makes any low-light image much darker than the original low-light image and the output can be seen its underexposed version. Likewise, for γ < 1, we have for CRT gamma (1): Therefore, we have Finally, from (6) and (10), we can get: As a result, the mapping defined by (5) makes any low-light image much brighter than GC and the output can be seen its overexposed version. From the above results and analysis, we can see that there is a complementary relationship between (3) and (5), so in order to better enhance low-light images and achieve a delicate balance, we can integrate them into an intrinsic whole and thus construct the following pair of complementary gamma functions (PCGF): We will hopefully obtain moderate exposed images with high quality by the above PCGF (12).

B. IMAGE ENHANCEMENT BY PCGF
By naïve blending strategy, we can derive the final enhanced image: where y i is the mapped image from the original low-light image according to (12) and the weighting factor is determined by: whereV i is the average value of mapped image y i in HSV space. Since either weighting factor is directly proportional to the average value component of the corresponding mapped image, the output image will have a moderate brightness. Figs. 4(a) and 4(b) show the corresponding enhanced images of those low-light images shown in Fig. 3(a) by PCGF which is performed on all RGB channels and only on the value component (V) in HSV space, respectively. In Fig. 4(a), all RGB channels of any original low-light image are mapped by PCGF (12). Then the enhanced image is obtained according to (13) and (14). While in Fig. 4(b), any low-light image is firstly transformed from RGB space to HSV space, the value component is mapped by PCGF (12), then two obtained value components are added according to (13) and (14) to output the value component in HSV space. Finally, the previous obtained value component (V), together with the hue component (H) and the saturation component (S) of the original low-light image, is transformed back to RGB space, and the desired enhanced RGB image is finally output.
It can be seen from Fig. 4 that mapping V in HSV space achieves better visual results than independently mapping all RGB channels. In fact, HSV color space provides a good color representation for human perception and can completely separate color information from brightness information, as means that the original color of any pixel will not be influenced by any processing of the value component [43]. Thus we will only treat the value component in HSV space during the fusion process in the next section.

IV. LOW-LIGHT IMGAE ENHANCEMENT BASED ON PCGF BY FUSION
From Fig. 4, it can be seen that PCGF can effectively enhance low-light images. It not only significantly improves the overall brightness of low-light images, but also enhances the details by stretching the dark regions and compressing the bright regions to a certain extent. However, the details and local contrast are still not prominent enough. To further improve the quality, we build on a simple fusion strategy. Image fusion is widely used for low-light image enhancement [23], [24], underwater image enhancement [32], image dehazing [44], multi-focus image processing [45], [46] and so on. Similar to [23] and [24], our approach is to blend a set of inputs and weight maps derived from a single original low-light image, which contains three main steps: deriving input images for fusion, defining weight maps, and fusing of the input images incorporating the defined weight maps.
The whole flow chart of proposed low-light image enhancement method based on PCGF by fusion is illustrated by Fig. 5. The value component (V) of an original low-light image is firstly processed by gamma enhancement according to (13) and by detail enhancement according to (19) which will be presented below, then two obtained value components are fused by the Laplace pyramids method shown as in (22) below. The value component (V) resulting from the fusion process and the hue component (H) and the saturation component (S) of the original low-light image are together transformed back to RGB space to output the desired enhanced RGB image.
In the following, we will show how to obtain input images for fusion and the corresponding weight maps and will also illustrate the fusion process in detail.

A. INPUT IMAGES FOR FUSION
The first input (I 1 ) for fusion is directly derived from the enhanced value component of the original low-light image in HSV space by PCGF according to (13) and (14).
The second input (I 2 ) for fusion is derived from the enhanced version of the original low-light image through combining sharpening and histogram equalization (HE). Image sharpening such as unsharp masking (UM) and classical HE are commonly used to improve details and contrast of low-light images, respectively. The typical formula for UM is as follows [39]- [41]: where I and S are the input image and sharpened image, respectively; G is a Gaussian kernel function; α is a preset constant to control the level of contrast enhancement which is selected between 0.5 and 1.5; * denotes the convolution operation. Some improved UM methods have been proposed in [40] and [41]. In [40], an adaptive sharpening strength selection method is given. While in [41] a pixel-wise method for enhancement strength is determined according to local blurriness measure of the original input image. However, if we only perform the sharpening procedure, image details will not be significantly enhanced due to the overall darkness of low-light images. In the following we will design a strategy to obtain the second input image for fusion by combining traditional UM method and HE method. Setting α = 1 in traditional UM (15), we can get: where V represents the value component of the original lowlight image in HSV space. However, UM itself isn't inadequate and doesn't work fine for low-light images, so we can improve the contrast by executing HE on V in (16) and hence we have: where H (·) denotes the HE operation. In order to further reduce artifacts, we can average the value component V and the above obtained S 2 and hence derive the second input image for fusion: Finally, we have the second input image for fusion:

B. WEIGHT MAPS FOR FUSION
Light exposure has great impact on images' quality. Therefore, we select exposedness weight map to obtain regions with good exposure from input images for fusion. Exposedness weight map is utilized in [23], [24] and [32] to evaluate the degree of exposure, which is expressed as a Gaussian-modeled distance to the average normalized range value 0.5 as follows: where I i (x, y) represents the ith input image with the pixel location (x, y). Fu et al. [23] conducted many experiments on 2000 images and found that the average of the mean values approximately equals 0.5 and the average of the standard deviations approximately equals 0.25. So the above exposedness weight map is reasonable. Higher values will be assigned to those pixels which are close to 0.5. This kind of weight map assigns small values to pixels with under-exposed and overexposed regions while normal-exposed pixels obtain high values. Hence, it achieves an unbiased assignment. Finally, the normalized weight maps for fusion are given by:

C. IMGAE FUSION PROCESS
In order to overcome undesirable halos and artifacts resulting from naive blending for fusion, we employ the pyramid fusion proposed by Burt and Adelson [35]. Both input images are decomposed into Laplace pyramids, while both corresponding weight maps are decomposed into Gaussian pyramids. Then blend Laplacian inputs and Gaussian weight maps of each level. Finally, the enhanced image is given by: VOLUME 8, 2020 where l represents the number of layers of the pyramids, U d is the up-sampling operator with d = 2 l − 1, G (·) and L (·) represent Gaussian operator and Laplace operator, respectively. By a large number of experiments, we find that decomposition into four layers can achieve excellent performance with low calculation load.

V. EXPERIMENTS AND DISCUSSION
In this section, we evaluate the performance of the proposed method. First, we present our experiment settings. Then, we evaluate the proposed method by comparing it with other state-of-the-art methods in both subjective and objective aspects.

A. EXPERIMENTS PREPARATIONS
To fully evaluate the proposed method, one hundred images with low illumination from various scenes have been tested. Test images come from Lee and Kim [6], Fu et al. [23], Wang et al. [24], Guo et al. [26], and our team. All images are processed in MATLAB R2017b on a PC running Windows 10 OS with 32G RAM and 2.1GHz CPU.

B. SUBJECTIVE EVALUATION
Three low-light images, which are named Street, Sea and Church are shown in Fig. 6(a) from left to right. Figs. 6(b) to 6(h) successively illustrate the enhanced results from CE [6], AGCWB [7], FE [23], BIMEF [25], LIME [26], SRM [27], LECARM [28] and the proposed PCGF method. Some zoom-in detailed patches are also shown on the right of each enhanced image. From Fig. 6, we can find that LIME causes color distortion; output images by FE are somewhat over-enhancement; the whole brightness by both The enhancement results of other five low-light images by the above different methods are shown in Fig. 7. We can find that enhanced images by CE are still a little dark and have poor visual quality. Enhanced images by AGCWD don't preserve details in bright regions, for example the crane's feather in the Crane image. BIMEF is effective in preserving the naturalness, but enhanced images are also a little dark. LIME can effectively improve the overall brightness. However, some regions such as the trees in the Walkway image and the Crane image suffer from over-enhancement. The details of enhanced image by SRM are not prominent enough. Our method achieves excellent visibility, contrast and details and the whole images have moderate brightness. While enhanced images by FE are a bit unnatural; and the enhanced images by LECARM are more or less over-exposed, whose details are slightly inferior to our method.

C. OBJECTIVE EVALUATION
Besides the subjective visual appearance and to further explain the effectiveness of our method, we employ NIQE [36], ILNIQE [37], SSEQ [38], and BRISQUE [42] to objectively evaluate the results of our method. These no-reference metrics are widely used for performance evaluation in the field of low-light image enhancement. A brief introduction is made for them in the following. NIQE (Natural Image Quality Evaluator) assesses image quality without any knowledge of distortions or human opinions of them through constructing several quality-aware  [6]; (c) AGCWB [7]; (d) FE [23]; (e) BIMEF. [25]; (f) LIME [26]; (g) SRM [27]; (h) LECARM [28]; (i) proposed PCGF.
statistical features based on the natural scene statistic model. ILNIQE (Integrated Local NIQE) can comprehensively capture local distortion artifacts through integrating qualityaware features by a local multivariate Gaussian model. SSEQ, which is a Spatial-Spectral Entropy-based Quality index, can assess distorted image's quality and achieve very good predictive performance. BRISQUE (Blind/Referenceless Image Spatial Quality Evaluator) quantifies naturalness losses resulting from distortions by using scene statistics of locally normalized luminance coefficients in the image. The smaller all these indexes, the better the performance. Besides, values for PSNR (Peak Signal-to-Noise Ratio) are also given.   [6]; (c) AGCWB [7];(d) FE [23]; (e) BIMEF [25]; (f) LIME [26]; (g) SRM [27]; (h) LECARM [28]; (i) proposed PCGF. It is obvious that larger value for PSNR means better quality of an enhanced image.
Quantitative results for total 100 images which consist of the same 92 random selected images from and 8 images shown in Figs. 6 and 7 are given in Table 1. It can be seen that our method has the best scores for NIQE, ILNIQE and SSEQ, which means our algorithm has absolute priority in low-light image enhancement to preserve the details, naturalness and reduce artifacts and distortions. Both scores for BRISQUE and PSNR by our method rank third out of eight. LECARM and CE have the best scores for BRISQUE and PSNR, respectively. Objectively speaking, our method can achieve satisfying scores. Overall, our method outperforms almost all other methods and hence is effective for low-light image enhancement.

VI. CONCLUSION
In this paper, we define a pair of complementary gamma functions (PCGF) by which we can obtain an underexposed version and an overexposed version of the original low-light image so that we can combine them to achieve a moderate brightness. We show the outstanding potential of PCGF for low-light image enhancement by some preliminary experimental results. We also design a fusion strategy to release its performance for low-light images based on PCGF. Two input images used for fusion are derived from the enhanced image by PCGF and the enhanced image by proposed sharpening method, respectively. Experiments show that proposed method can significantly enhance the detail and improve the contrast of low-light image. Objective and subjective assessment between different methods show its outperformance. As future work, we plan to develop better fusion methods to achieve its full potential. Besides, we will try to apply the proposed PCGF for foggy image dehazing, underwater image enhancement and saliency detection and so on.