An Experiment-Based Review of Low-Light Image Enhancement Methods

,


FIGURE 1. Examples of low-light images.
the X-axis shows the grayscale values and the Y-axis represents the number of pixels. The pixel values of these images are mainly focused in the lower range due to the lack of illumination, and the gray difference of the corresponding pixels between the various channels of the color image is limited. There is only a small gap between the maximum and minimum gray levels of the image. The whole color layer exhibits deviations, and the edge information is weak; consequently, it is difficult to distinguish details of the image. These characteristics reduce the usability of such images, seriously degrade their subjective visual effect, and greatly limit the functionality of various visual systems [16]- [18].
To weaken the impact of video/image acquisition from low-illumination environments, researchers have pursued various improvements from both the hardware and software perspectives. One approach is to improve the image acquisition system hardware [19]- [21]. Another is to process the images after they are generated. Because low-illumination cameras use high-performance charge-coupled device (CCD) or complementary metal-oxide-semiconductor (CMOS) technology, professional low-light circuits, and filters as the core components to improve the imaging quality for lowlight-level imaging, their manufacturing process is highly rigorous, and the technology is complex [22]. Although some professional low-light cameras produced by companies such as Sony, Photonis, SiOnyx and Texas Instruments (TI) have appeared on the market, they are not widely used in daily life because of their high prices. As an alternative approach, the improvement of software algorithms offers great flexibility, and improving the quality of low-light videos and images by means of digital image processing has always been an important direction of research. Therefore, it is of great significance and practical value to study enhancement algorithms for low-light images to improve the performance of imaging devices.
The main purpose of low-light image enhancement is to improve the overall and local contrast of the image, improve its visual effect, and transform the image into a form more suitable for human observation or computer processing, while avoiding noise amplification and achieving good real-time performance. [23]- [27]. To this end, it is essential to enhance the validity and availability of data captured under low illumination to obtain clear images or videos [28]. Such enhancement can not only render images more consistent with the subjective visual perception of individuals and improve the reliability and robustness of outdoor visual systems but also allow such images to be more conveniently analyzed and processed by computer vision equipment, which is of great importance for promoting the development of image information mining [29], [30]. Related research results can be widely applied in fields such as urban traffic monitoring, outdoor video acquisition, satellite remote sensing, and military aviation investigation and can be used as a reference for studies on topics such as underwater image analysis and haze image clarity [31]. Moreover, as an important branch of research in the field of image processing, low-light image enhancement has interdisciplinary and innovative appeal and broad application prospects and has become a focus of interdisciplinary research in recent years [32]. A large number of researchers at home and abroad have been paying increasing attention to this field for quite some time [33]- [38].
In the real world, color images are most commonly used, so most of the algorithms are either designed for color image enhancement or derived from gray image enhancement methods. The major methods are listed below.
(i) Enhancement based on the RGB (red, green, blue) color space. The specific steps are as follows. The three color components (R, G and B) are extracted from the original RGB color image. Then, these three components are each individually enhanced using a grayscale image enhancement method. Finally, the three components are merged, and the enhanced results are output. The specific principle is visually summarized in Fig. 2. This method is simple but can result in serious color deviations in the enhanced images because it neglects the correlations between the components. (ii) Enhancement based on the HSI (hue, saturation, intensity) color space (or the YCbCr, L * a * b, YUV color space) [39]- [42]. The brightness component I in the HSI color space is separate from and unrelated to the chrominance component H , i.e., the color information of an image. When the chrominance does not change, the brightness and saturation will determine all of the image information. Hence, to enhance a color image, the I and S components are usually enhanced separately while maintaining the same chromaticity H . Fig. 3 shows the flow chart of this enhancement approach. In recent years, a common method used to process color images has been to leave one component unchanged while enhancing the other components based on a space transformation. Notably, the available transformations of the color space are diverse, and the form of an image is not limited to a certain space. Regardless of the color space used, the processing steps are similar to those of an enhancement method based on the HSI space [43]- [45]. Studies on low-light image enhancement technology are currently still being conducted by many researchers. Although promising findings have been obtained, this technology is still not mature. In particular, the available algorithms often have a better effect in a certain aspect than in others. Thus, the field still has significant research value and offers a large development space that is attractive to researchers.
The remainder of this paper is organized as follows. Section II introduces a classification of low-light image enhancement algorithms according to their underlying principles, and the characteristics of the different categories of algorithms are analyzed in detail. In Section III, related quality assessment criteria for enhanced images are described. Several experiments implemented to test the performance of the representative methods are described in Section IV, and the conclusions are summarized and future research directions suggested in the last section.

II. CLASSIFICATION OF LOW-LIGHT IMAGE ENHANCEMENT ALGORITHMS
Scholars worldwide have proposed many image enhancement algorithms for images captured under low-illumination conditions to improve low-light videos and images from different perspectives [1], [3], [7]. In accordance with the algorithms used for brightness enhancement, this paper divides these processing methods into seven classes: gray transformation methods, histogram equalization (HE) methods, Retinex methods, frequency-domain methods, image fusion methods, defogging model methods and machine learning methods. These methods can be further divided into different subclasses in accordance with the differences in their principles. The overall classification is depicted in Fig. 4.

A. GRAY TRANSFORMATION METHODS
A gray transformation method is a spatial-domain image enhancement algorithm based on the principle of transforming the gray values of single pixels into other gray values by means of a mathematical function [46], which is usually called a mapping-based approach. Such a method enhances an image by modifying the distribution and dynamic range of the gray values of the pixels [1], [7]. The main subclasses of this type method include linear and nonlinear transformations.

1) LINEAR TRANSFORMATION
A linear transformation of gray values, also known as a linear stretching, is a linear function of the gray values of the input image [1], and the formula is as follows: g(x, y) = C · f (x, y) + R (1) where f (x, y) and g(x, y) represent the input and output images, respectively, and C and R are the coefficients of the linear transformation. An image can be enhanced to different degrees by adjusting the values of the coefficients in the above formula. A corresponding transformation curve is shown in Fig. 5(a). A common formula for a linear gray stretch is as follows: g(x, y) = f (x, y) − f min f max − f min (g max − g min ) + g min (2) where f max and f min represent the maximum and minimum gray values of the input image, respectively, and g max and g min represent the maximum and minimum gray values of the output image, respectively [7]. Thus, the dynamic range of the image is transformed from [f min , f max ] to [g min , g max ] to enhance the brightness and contrast. Sometimes, the gray values in only a specific area of the image need to be stretched 87886 VOLUME 8, 2020 or compressed by applying a piecewise linear transformation to adjust the contrast; the formula for such a transformation is as follows.
The various functions in the piecewise formula are represented by colored polylines in the coordinate system corresponding to the transformation. The positions of the discontinuity points must be determined individually for each specific image. An example of such a piecewise linear transformation curve is shown in Fig. 5 In the piecewise linear transformation method, parameter optimization can be performed only based on experience or with considerable human participation; thus, it lacks an adaptive mechanism. Additionally, it is difficult to achieve the optimal enhancement effect [46], [47], [47]- [49]. To overcome these problems, a hybrid genetic algorithm combined with differential evolution has been applied for image enhancement processing [50]. The optimal transformation curve was obtained through the adaptive mutation and quick search capabilities of this algorithm. In summary, the principle of linear image enhancement is simple and fast to execute, but the effect is not satisfactory, with some image details typically being lost due to uneven image enhancement.

2) NONLINEAR TRANSFORMATION
The basic idea of a nonlinear gray transformation is to use a nonlinear function to transform the gray values of an image [51]. Frequently used nonlinear transformation functions include logarithmic functions, gamma functions and various other improved functions [52], [53]. A logarithmic transformation function implies that there is a logarithmic relationship between the value of each pixel in the output image and the value of the corresponding pixel in the input image. This type of transformation is suitable for an excessively dark image because it can stretch the lower gray values of the image while compressing the dynamic range of the pixels with higher gray values [54]. The typical formula is as follows: where c is a control parameter. The shapes of several logarithmic transformation functions are shown in Fig. 6(a). A logarithmic transformation function stretches the gray values of pixels in low-gray-value areas and compresses the values of pixels in high-gray-value areas. The gamma function is a nonlinear transformation with broad application whose formula is as follows: where γ denotes the gamma correction parameter, which is usually a constant. Several gamma transformation curves are shown in Fig. 6(b). As shown in Fig. 6(b), several different transformation curves can be obtained by varying the parameter γ . When γ > 1, the transformation will stretch the dynamic range of the low-gray-value areas of the image and compress the range of the high-gray-value areas. In contrast, when γ < 1, the transformation will compress low gray values and stretch high gray values. When γ = 1, the output remains the same [55]. Therefore, different gray regions of an image can be selectively stretched or compressed by adjusting this parameter to obtain a better enhancement effect. Drago et al. suggested that the dynamic range of an image can be effectively compressed by mapping the gray values of the image using an adaptive logarithmic function [56]. Tao et al. used a gray value corresponding to the cumulative histogram of an image with a value of 0.1 to self-adaptively obtain a nonlinear mapping function that can enhance the brightness of dark regions while inhibiting the enhancement of bright regions [57]. Likewise, a low-light color image enhancement algorithm based on a logarithmic processing model was proposed by Tian et al. [58]. First, this algorithm applies a nonlinear enhancement process to the brightness components of an image; then, the membership function is amended by introducing an enhancement operator. Finally, the enhanced image is obtained by means of the inverse transform of the membership function. Huang et al. [59] proposed an adaptive gamma correction algorithm in which the gamma correction parameter is adaptively obtained in accordance with the cumulative probability distribution histogram. However, because this gamma correction method relies on a single parameter, it is prone to cause overenhancement of bright areas. To overcome this shortcoming, a double gamma function was constructed by Zhi et al. and adjusted based on the distribution characteristics of the illumination map [60], thus improving the gray values in low-brightness areas while suppressing the gray values in local high-brightness regions. Moreover, an arctangent hyperbola has been used to map the hue component of an image to an appropriate range by Yu et al. [61], and later, low-light image enhancement based on the optimal hyperbolic tangent profile was proposed [62]. Nonlinear transformation requires more complex calculations and consequently a longer time than linear transformation [63], [64].
In summary, gray transformation can highlight gray areas of interest and has the advantages of simple implementation and fast speed. However, such methods do not consider the overall gray distribution of an image; consequently, their enhancement ability is limited, and their adaptability is poor.

B. HISTOGRAM EQUALIZATION (HE) METHODS
If the pixel values of an image are evenly distributed across all possible gray levels, then the image shows high contrast and a large dynamic range. On the basis of this characteristic, the HE algorithm uses the cumulative distribution function (CDF) to adjust the output gray levels to have a probability density function that corresponds to a uniform distribution; in this way, hidden details in dark areas can be made to reappear, and the visual effect of the input image can be effectively enhanced [65], [66].

1) PRINCIPLE OF HE
In the HE method, the CDF is used as the transformation curve for the image gray values [67]- [71]. Let I and L denote an image and its gray levels, respectively. I (i, j) represents the gray value at the position with coordinates (i, j), N represents the total number of pixels in the image, and n k represents the number of pixels of gray level k. Then, the gray-level probability density function of image I is defined as The CDF of the gray levels of image I is The standard HE algorithm maps the original image to an enhanced image with an approximately uniform gray-level distribution based on the CDF. The mapping relationship is as follows: An example of HE is shown in Fig. 7, in which (a) presents the input low-light image, (b) displays the histogram of the input low-light image, (c) presents the enhanced image after HE, and (d) displays the histogram of the enhanced image. The principle of the standard HE algorithm is simple and can be executed in real time. However, the brightness of the enhanced image will be uneven, and some details may be lost due to gray-level merging.

2) BASIC MODELS OF HE METHODS
Depending on the regions considered in the calculation, HE methods can be divided into global histogram equalization (GHE) and local histogram equalization (LHE) [72]. The general concept of a GHE algorithm is illustrated by the model shown in Fig. 8, where X represents the original image, Y represents the enhanced image generated by the HE algorithm, Y = f (X ) represents the traditional HE process or an improved version, and X 1 , X 2 , X 3 , · · · , X n represent n subimages composed of pixels in the original image that satisfy certain conditions according to a given property, which is defined as Q(x). The parameter x represents the magnitude of the image gray value, Y 1 , Y 2 , Y 3 , · · · , Y n denote the equalized images corresponding to the n subimages, and the image Y after equalization is obtained by merging the subimages in accordance with the pixel positions.
The GHE model has several advantages, such as relatively few calculations and high efficiency, and it is especially suitable for the enhancement of overall darker or brighter images [58]. However, it is difficult for a global algorithm, which conducts statistical operations based on the gray values of the whole image, to obtain the optimal recovered values for each local region. Such an algorithm is unable to adapt to the local brightness characteristics of the input image, and consequently, the sense of depth in the image will be decreased after processing. To solve this problem, many scholars have proposed that an LHE algorithm should be used instead, and such algorithms have hence been put into wide practice. The basic idea of LHE is to apply the HE operation separately to various local areas of an image. The original image is spatially divided into multiple subblocks, and equalization is conducted separately on each subblock to adaptively enhance the local information of the image to achieve the desired enhancement effect. LHE methods can be further divided into three approaches [72], namely, LHE with nonoverlapping subblocks, LHE with overlapping subblocks and LHE with partially overlapping subblocks, as shown in Fig. 9. The implementation process for these algorithms is as follows.
(i) For an input image of a given size, M × N , a subblock with dimensions of m × n is defined in the upper left corner of the image, and additional subblocks are defined by moving along the horizontal and vertical directions with step sizes of h and w, respectively.
(ii) HE processing is applied to each subblock in the same manner as for a GHE algorithm.
Then, the results are added to the output image, and the cumulative number of subblock processing rounds for each pixel is recorded.
(iii) The next subblocks are defined by moving horizontally with the horizontal step size h and vertically with the vertical step size w. For each subblock that does not exceed the image boundary, step (ii) is repeated; if no such unprocessed subblocks remain, the method proceeds to the next step.
(iv) The output image is obtained by dividing the gray value of each pixel in the output image by the corresponding cumulative number of subblock processing rounds. The disadvantages of this algorithm are the local block effects and the large number of calculations.

3) HE ALGORITHMS
Many algorithms have been developed based on the classic HE approach. For example, Kim first proposed brightness-preserving bi-histogram equalization (BBHE) to maintain the image brightness [73], in which the input image is divided into two subimages I L and I U (satisfying the conditions I = I L ∪I U and I = I L ∩I U = ) using the mean brightness of the original image as a threshold. HE is then applied to each subimage to address the issue of uneven brightness in local areas of the enhanced image. Subsequently, Wang et al. proposed the equal-area dualistic subimage histogram equalization (DSIHE) algorithm [74]. This algorithm uses the median gray value of the original image as a threshold to divide the image into two parts of the same size to maximize its entropy value, thus overcoming the loss of image information caused by the standard HE algorithm. Later, Chen proposed the minimum mean brightness error bi-histogram equalization (MMBEBHE) model [75], which minimizes the mean brightness error between the output image and the original image. Furthermore, Shen et al. proposed the iterative brightness bi-histogram equalization (IBBHE) algorithm [76], in which the segmentation threshold is selected through an iterative method to drive the mean to converge to the optimum while avoiding the confusion between target and background that can occur in traditional HE. Similarly, a BBHE algorithm that preserves color information was proposed by Tian et al. [77]. This algorithm not only retains the color information of the input image but also enhances the image details. Other optimization methods, including local approaches, have also been continuously emerging. For example, a standard adaptive histogram equalization (AHE) algorithm was proposed in [78], while in [79], a block iterative histogram method was used to enhance the image contrast, and a moving template was used for partially overlapped subblock histogram equalization (POSHE) processing for each part of the image. Liu et al. [80] proposed an LHE method that uses a histogram-number-based gray-level protection mechanism on the basis of nonoverlapping subblocks. The spatial positions of the pixels in each block are taken into account when setting the weights, thus effectively eliminating the block effect. Huang and Yeh [81] proposed a novel LHE algorithm that can improve the contrast of an image while maintaining its brightness. Reza [82] proposed the contrastlimited adaptive histogram equalization (CLAHE) algorithm. This algorithm effectively mitigates the block effect that arises in the enhancement process and limits local contrast enhancement by setting a threshold, thus avoiding excessive enhancement of the image contrast. The CLAHE method can be combined with the Wiener filter (WF) or a finite impulse response filter (FIRF) for image contrast enhancement, as discussed in [83] and [84], respectively. Based on BBHE and recursive mean-separate histogram equalization (RMSHE) [85], an LHE algorithm that maintains image brightness was proposed in [86]. In this algorithm, RMSHE is applied to each subblock, thus effectively maintaining the mean brightness of the subblocks. Based on DSIHE and RMSHE, a POSHE method with equal-area recursive decomposition was proposed in [87]. In this algorithm, multiple equal-area recursive decompositions are implemented on the VOLUME 8, 2020 subblocks in combination with DSIHE and RMSHE to more effectively maintain the image brightness compared with DSIHE. Simultaneously, a bilinear difference measure is used to eliminate the differences in contrast between each subblock and the two externally adjacent subblocks, thus improving the visual effect of the image. In addition, an LHE algorithm based on entropy maximization and brightness maintenance was proposed in [88] that maximizes the entropy of the subblocks while leaving their mean brightness unchanged before and after equalization, thus effectively enhancing the image details. In [89], the contextual and variational contrast (CVC) enhancement algorithm was proposed; in this algorithm, the two-dimensional histogram and context information model of the input image are used to implement nonlinear data mapping to enhance a weakly lighted image. In [59], an HE method with gamma correction was proposed and achieved a balance between the quality of the output image and the computing time. However, the HE methods discussed above typically fail to effectively eliminate the potentially severe interference of noise in weakly illuminated images; in fact, they may even amplify such noise. Therefore, researchers have proposed interpolationbased HE algorithms, in which linear interpolation methods are used to determine the transformation function for the current pixels, thus overcoming the "block effect" caused by nonoverlapping subblocks in HE and achieving a better enhancement effect. In recent years, the newly proposed algorithms have all been combined with image analysis. The background brightness-preserving histogram equalization (BBPHE) algorithm [90] divides the input image into the background region and the target region, while the dominant orientation-based texture histogram equalization (DOTHE) algorithm [91] divides the image into textured and smooth regions. Other typical algorithms include gain-controllable clipped histogram equalization (GCCHE) [72], recursive subimage histogram equalization (RSIHE) [92], entropybased dynamic subhistogram equalization (EDSHE) [93], dynamic histogram equalization (DHE) [94], brightnesspreserving dynamic histogram equalization (BPDHE) [95], bi-histogram equalization with a plateau limit (BHEPL) [96], median-mean-based subimage-clipped histogram equalization (MMSICHE) [97], exposure-based subimage histogram equalization (ESIHE) [98], adaptively modified histogram equalization (AMHE) [99], weighted histogram equalization (WHE) [100], a histogram modification framework (HMF) [101], gap adjustment for histogram equalization (CegaHE) [102], and unsharp masking with histogram equalization (UMHE) [103].
To illustrate the performance of HE methods on color images, the AMHE [99], BBHE [73], CLAHE [82], DSIHE [74], HE [66], RMSHE [85], RSIHE [92], and WHE [100] algorithms are tested here in both the RGB and HSI color spaces. The test image and its results are shown in Figs. 10-12. (i) Equalization and merging of the three R, G and B subimages   The contrast of gray images can be effectively enhanced; however, for a color image with three components (R, G and B), serious color distortion of the image may be observed if the final image is obtained by simply equalizing and merging the R, G, and B subimages after equalization. The main reason is that the traditional HE algorithm excessively enhances the image brightness. If the mean brightness of one of the three R, G and B subimages of a color image is too dark or too bright, then the mean brightness of this subimage after equalization will be near the median gray value of this component. As a result, the color corresponding to this subimage will be either strengthened or weakened after enhancement, resulting in obvious color distortion and inconsistency in the final color image. Therefore, the primary goal of HE for a color image using this method is to maintain the mean brightness of the image while enhancing the image contrast.
(ii) In the HSI model, only the brightness component is equalized. In this method, the input color image is first converted from the RGB color space to the HSI color space, and then HE enhancement is applied to the brightness component I. Finally, the color image is converted back to the RGB space. In this way, the number of equalizations is reduced from 3 to 1. However, some calculations are still needed for the transformation between the color spaces, and there is still a risk of excessive image enhancement.
In summary, HE algorithms can effectively enhance lowlight images and are often used in combination with other methods. The visual effect of such an image can be improved based on the contrast and detail enhancement provided by an HE algorithm. However, these methods can also easily cause a loss of color fidelity and the generation of noise, resulting in image distortion.

C. RETINEX METHODS
The Retinex theory, namely, the theory of the retinal cortex, established by Land and McCann, is based on the perception of color by the human eye and the modeling of color invariance [104]. The essence of this theory is to determine the reflective nature of an object by removing the effects of the illuminating light from the image. According to Retinex theory, the human visual system processes information in a specific way during the transmission of visual information, thus removing a series of uncertain factors such as the intensity of the light source and unevenness of light. Consequently, only information that reflects essential characteristics of the object, such as the reflection coefficient, is retained [105]- [109]. Based on the illumination-reflection model (as shown in Fig. 13), an image can be expressed as the product of a reflection component and an illumination component [110]: where R(x, y) is the reflection component, which represents the reflective characteristics of the object surface; L(x, y) is the illumination component, which depends on the environmental light characteristics; and I (x, y) is the received image. L(x, y) determines the dynamic range of the image, whereas R(x, y) determines the inherent nature of the image. According to Retinex theory, if L(x, y) can be estimated from  I (x, y), then the reflection component can be separated from the total amount of light, and the influence of the illumination component on the image can be reduced to enhance the image [111]. The Retinex algorithm features a sharpening capability, color constancy, large dynamic range compression and high color fidelity. The general process of the Retinex algorithm is shown in Fig. 14, where Log denotes the logarithmic operation and Exp denotes the exponential operation. Many researchers have proposed effective image enhancement algorithms based on the Retinex theory. First, Land proposed that the illumination component could be estimated by using a random path algorithm to reduce the effect of uneven illumination. However, this random path algorithm is complex and has a common effect. Later, a two-dimensional path selection method, namely, the central Retinex algorithm, was proposed. Its core idea is as follows: an appropriate surround function is selected to determine the weighting of the pixel values in the neighborhood of the current pixel, which are then used to replace the current pixel value. Subsequently, Jobson et al. proposed the single-scale Retinex (SSR) algorithm [112], [113], followed by the multiscale Retinex (MSR) algorithm and the multiscale Retinex algorithm with color restoration (MSRCR) [114], [115].

1) SINGLE-SCALE RETINEX (SSR)
Essentially, the SSR algorithm obtains a reflection image by estimating the ambient brightness. The formula is as follows: (10) where I (x, y) represents the input image, R(x, y) represents the reflection image, i represents the various color channels, (x, y) represents the position of a pixel in the image, G(x, y) represents the Gaussian surround function, and * represents the convolution operator.
The formula for the Gaussian surround function is where σ is a scale parameter. The smaller the value of this parameter is, the larger the dynamic range compression of VOLUME 8, 2020 the image is, and the clearer the local values are. K is a normalization factor to ensure that the Gaussian function satisfies To better mimic the characteristics of the human visual system, automatic gain compensation is often required; that is, the output image is mapped to [0, 255] using a linear gray stretching algorithm. The mathematical formula is as follows: where R i (x, y) is the output after gray stretching of the i th color channel, and R max and R min are the maximum and minimum gray levels of the original image, respectively. Fig. 15 shows the enhancement results obtained using the SSR method when σ is 15, 80 and 250, respectively. However, the SSR algorithm has some limitations. It is difficult to maintain a balance between detailed information enhancement and color fidelity in images processed with this algorithm due to the use of a single scale parameter.

2) MULTISCALE RETINEX (MSR)
To maintain a balance between dynamic range compression and color constancy, Jobson, Rahman et al. extended the single-scale algorithm to a multiscale algorithm, namely, the MSR algorithm [114], which is expressed as follows: where i represents the three color channels; k represents the Gaussian surround scales; N is the number of scales, generally 3; and the ω parameters are the scale weights. Compared with the SSR algorithm, the MSR algorithm can take advantage of the benefits of multiple scales. The MSR algorithm not only enhances image details and contrast but also produces enhanced images that exhibit better color consistency and an improved visual effect.

3) MULTISCALE RETINEX WITH COLOR RESTORATION (MSRCR)
During the process of image enhancement, the SSR or MSR algorithm is applied separately to the three color channels, R, G and B. Therefore, compared with the original image, the relative proportions of the three color channels may change after enhancement, thus resulting in color distortion.
To overcome this problem, MSRCR has been proposed. This algorithm includes a color recovery factor C for each channel, which is calculated based on the proportional relationship among the three color channels in the input image and is then used to correct the color of the output image to eliminate color distortion.
The formula for the color recovery factor is as follows: where f denotes the mapping function, and C(x, y) is the color recovery factor. Jobson et al. found that the best color recovery effect is achieved when the mapping function is a logarithmic function, namely, The mathematical expression for the MSRCR algorithm can be obtained by combining formulas (18) and (15): The algorithm takes advantage of the convolution operation with Gaussian functions. Dynamic range compression and color constancy are achieved for features at large, medium and small scales, thus yielding a relatively ideal visual effect. Experimental results obtained with the SSR, MSR and MSRCR algorithms are shown in Fig. 16.

4) OTHER RETINEX ALGORITHMS
Retinex theory conforms to the characteristics of human visual perception; consequently, it has been widely applied and developed. Many enhancement algorithms have been proposed based on Retinex theory [116]- [119]. Kimmel et al. used the transcendental hypothesis to propose a Retinex algorithm based on a variational framework [120], in which the problem of illumination estimation is transformed into an optimal quadratic programming problem. Despite its high complexity, this algorithm achieves good results. Elad et al. proposed a noniterative Retinex algorithm that can process the edges in an image and suppress noise in dark areas [121]. Meylan et al. proposed a new model for presenting images with a high dynamic range and adapting images both globally and locally to the human visual system. In this algorithm, an adaptive filter is applied to reduce the chromatic aberrations caused by halo effects and brightness modification [122]. Xu et al. proposed a rapid Retinex image enhancement method that eliminates the halo phenomenon encountered with the traditional Retinex algorithm in areas of high light and dark contrast [123].
Likewise, Marcelo et al. proposed a kernel-based Retinex (KBR) method, which relies on calculating the expected value of a suitable random-variable-weighted kernel function to reduce color error and improve details in a shadow image [124]. In 2011, Ng et al. proposed a total variation model based on the Retinex algorithm; in this model, the illumination component is spatially smooth, and the reflection component is piecewise continuous. Moreover, a fast calculation method was used to solve the minimization problem posed by the variation model. Finally, the validity of the proposed model was verified through experiments [125]. In addition, Fu et al. proposed a weighted variation model for the simultaneous reflectivity and illumination estimation (SRIE) of observed images; this model can precisely retain the estimated reflectivity while inhibiting noise to some extent [126]. Petro et al. proposed a multiscale Retinex algorithm with chromaticity preservation (MSRCP) [127]. First, the image brightness data are processed using the MSR algorithm; then, the results are mapped to each channel in accordance with the original proportional relationship among the R, G and B channels. Thus, the image is enhanced while retaining the original color distribution, and the grayish color typically observed in images enhanced using the MSRCR algorithm is effectively improved [128]. Later, Matin et al. optimized the MSRCP method using particle swarm optimization (PSO) to avoid manual adjustment of the parameters [129]. Chen and Beghdadi [130] proposed an image enhancement algorithm based on Retinex and a histogram stretch method to maintain the natural color of images. Shen and Hwang [131] proposed an image enhancement algorithm based on Retinex with a robust envelope. To avoid color distortion, Jang et al. [132] proposed an image enhancement algorithm based on the use of MSR to estimate the main color of an image. Inspired by Retinex theory, Wang et al. [133], Xiao et al. [134] used a bionic method to enhance images. Chang Hsing Lee et al. proposed an adaptive MSR algorithm [135] based on brightness classification. For pixels in dark areas and bright areas, higher weights were given to larger-scale SSR components to enhance the overall visual effect of the image. Wang et al. [136] proposed a bright-pass filter in combination with neighborhood brightness information to maintain image naturalness, not only improving the image contrast but also better maintaining natural brightness without requiring naturalness preserved enhancement (NPE). Based on Retinex theory, some scholars have separated the reflection and illumination components and then enhanced the latter using a local nonlinear transformation model to render a brighter and more natural image [137]. As an alternative, an enhancement adjustment factor has been introduced [138] to adjust the enhancement degrees of different brightness values to avoid noise amplification and color distortion. Fu et al. [139] proposed a Retinex algorithm based on a variation framework that effectively enhances the contour details of an image while suppressing abnormal enhancement. In 2014, Zhao proposed a Retinex algorithm based on a Markov random field model. This algorithm estimates the illumination component of an image by means of guided filtering and solves for the reflection component of the object of interest based on the Markov random field model; additionally, this algorithm solves problems such as detail loss, color distortion and halo effects encountered when the MSRCR algorithm is used to process nighttime color images [140]. Later, Zhao et al. [141] proposed a Retinex algorithm based on weighted least squares. Jae et al. [142] proposed an MSR algorithm based on subband decomposition with a fusion strategy. Moreover, Liu et al. [143] combined the Retinex algorithm with bilateral filtering, thereby effectively improving the color distortion and detail loss in the final image but also increasing the complexity of the algorithm [144], [145]. Yin et al. [146], Mulyantini and Choi [147], Zhang et al. [148], Ji et al. [149], and Zhang et al. [150] proposed Retinex-based algorithms combined with guided filters [151]. Particularly during the early stage of research on Retinex algorithms, scholars obtained many fruitful findings.
In short, Retinex algorithms have clear benefits and can be easily implemented. These methods can not only increase the contrast and brightness of an image but also has obvious advantages in terms of color image enhancement. However, these algorithms use the Gaussian convolution template for illumination estimation and do not have the ability to preserve edges; consequently, they may lead to halo phenomena in some regions with sharp boundaries or cause the whole image to be too bright.

D. FREQUENCY-DOMAIN METHODS
With the development of multiscale image analysis technology, research on image enhancement algorithms has been extended from the spatial domain to the frequency domain [152]. Image enhancement methods based on the frequency domain transform an image into the frequency domain for filtering by means of Fourier analysis, and the final image is then inversely transformed back into the spatial domain. Typical frequency-domain methods include homomorphic filtering (HF) and wavelet transform (WT) methods. VOLUME 8, 2020

1) HOMOMORPHIC FILTERING (HF)
HF-based enhancement methods use the characteristics of the illumination-reflection model to transform the illumination and reflection components in the form of a sum in the logarithmic domain rather than a product. A high-pass filter is used to enhance the high-frequency reflection component and suppress the low-frequency illumination component in the Fourier transform domain [7].
The specific steps of the HF process are listed as follows.
(i) In the illumination-reflection model, the illumination component is multiplied by the reflection component, which cannot be transformed into the frequency domain. Therefore, to allow these components to be processed separately, the logarithmic transformation should first be implemented to transform these multiplicative components into additive components. Taking the logarithm of both sides of equation (10) yields the following: (ii) The image is transformed from the spatial domain into the frequency domain by means of the Fourier transform, i.e., the Fourier transform is applied to both sides of the above equation: This equation can be written more concisely as where I (u, v), L(u, v) and R(u, v) are the Fourier transforms of I (u, v), L(u, v) and R(u, v), respectively. The spectral function L(u, v) is mainly concentrated in the low-frequency range, while the spectral function R(u, v) is mainly concentrated in the high-frequency range.
(iii) For contrast enhancement, an appropriate high-pass filter is selected, and the R(u, v) component in the frequency domain is enhanced by the transfer function H (u, v). The resulting expression is as follows: (iv) The inverse Fourier transform is used to transform the image from the frequency domain back into the spatial domain. Let s(u, v) denote the inverse Fourier transform corresponding to S(u, v); then, the inverse Fourier transform of equation (10) is Therefore, the enhanced image corresponds to the superposition of the illumination component and reflection component.
(v) The inverse logarithmic transform G(x, y) = exp[s(x, y)] is applied to equation (24) to obtain the final corrected image. Thus, by taking the exponent of both sides of equation (24), the image after frequency-domain correction is obtained as follows: Therefore, the core of the HF technique is to design an appropriate filter H (u, v) based on the image properties characterized by the illumination component and the reflection component in combination with a frequency filter and a gray transformation to compress the dynamic range and enhance the contrast. A homomorphic filter has the following general form: where γ L < 1 and γ H < 1; the purpose of these parameters is to control the scope of the filter amplitude. H hp is usually a high-pass filter, such as a Gaussian high-pass filter, a Butterworth high-pass filter, or a Laplacian filter. If a Gaussian filter is used as H hp , then where c is a constant that controls the form of the filter. The larger the value of the transition gradient from low frequency to high frequency is, the steeper the slope, as shown in Fig. 17.  The specific algorithm flow of the HF method is shown in Fig. 18. In this figure, Log is the logarithmic transform, FFT is the fast Fourier transform, H (u,v) is the frequency filtering function, IFFT is the inverse FFT, and Exp is the exponential operation.
The traditional HF algorithm requires two Fourier transforms and thus is not suitable for real-time processing. To address this issue, some scholars have proposed an HF algorithm based on a spatial filter [153], [154]. The main idea is similar to that of the traditional HF algorithm. First, the original image is transformed into the logarithmic domain, and then, the output of a low-pass filter is used to estimate the illumination component. Finally, the reflection component is added to obtain the enhanced image. Because the traditional HF algorithm does not account for the local features of the image space, Zhang and Xie [155] proposed an HF algorithm based on the block discrete cosine transform (DCT) and removed the block effect after HF by considering the average boundaries with adjacent subimages as well as the characteristics of the DCT. Images processed with this algorithm show a good effect in terms of local contrast. A two-channel HF color image enhancement method based on the HSV color space was proposed in [41] by Han Lina et al. First, the input color image is transformed from RGB space into HSV space, thus obtaining separate chroma-, saturation-and brightness-channel images. Then, the saturation (S)-channel image is enhanced via Butterworth HF, and the brightness (V)-channel image is enhanced via Gaussian HF. Finally, the image is transformed back into RGB space to obtain the enhanced image. Fig. 19 shows the effects of enhancement processing with a Gaussian highpass filter and presents the histograms corresponding to the images. The image brightness is improved after the HF process, but the image details are fuzzy. Each coin has two sides: HF has the advantage of better maintaining the original image content, but its disadvantage is that it requires two Fourier transforms, namely, one exponential operation and one logarithmic operation, and therefore involves more calculation [156]- [158]. If the cut-off frequency of the high-pass filter is too high, then the dynamic range will be compressed and details will be lost. If the cutoff frequency is too low, the dynamic range compression will be minimal, and the algorithm will lack self-adaptability. This method is based on the premise of uniform illumination; consequently, the enhancement effect is poor for nighttime images with both bright and dark areas.
HF algorithms can remove uneven regions generated by light while maintaining the contour information of an image. However, such an algorithm requires two Fourier transformations, i.e., one exponential operation and one logarithmic operation, for each pixel in an image; therefore, its computational burden is large.

2) WAVELET TRANSFORM (WT)
Similar to the Fourier transform, the WT is a mathematical transform that uses a group of functions called a wavelet function basis to represent or approximate a signal [159]. The WT can be used not only to characterize the local features of signals in the time and frequency domains but also to conduct a multiscale analysis of functions or signals through operations such as scaling and translation. Thus, great progress in image contrast enhancement has been achieved using WT methods. In a WT-based image enhancement algorithm, the input image is first decomposed into low-frequency and high-frequency image components; then, image components at different frequencies are separately enhanced to highlight the details of the image. The main idea of the wavelet analysis method is to apply wavelet decomposition to the original image to obtain the wavelet coefficients for different subbands, adjust these wavelet coefficients, and then apply the inverse transformation to the new coefficients to obtain the processed image. Such an image enhancement algorithm can enhance an image at multiple scales based on the WT. It is believed that low-illumination conditions have a greater influence on high-frequency image components, which are generally concentrated at the edges of an image and in contour regions [160]. Therefore, a WT-based algorithm will enhance the high-frequency components of the input image and suppress its low-frequency components. In particular, the dual-tree complex WT can usually achieve satisfactory results [161]- [165].
The basic process of WT-based image enhancement is as follows. Processing of the displacement τ is carried out for the function ψ(t) describing the basic wavelet (parent wavelet); then, a wavelet sequence can be obtained by taking the inner product between the processed ψ(t) and the signal x(t) to be analyzed at various scales a.
The equivalent expression in the frequency domain is where X (ω) and ψ(ω) represent the Fourier transforms of x(t) and ψ(t), respectively. In standard WT-based image enhancement, the input image is usually first decomposed into one low-pass subimage and three directional high-pass subimages, namely, a horizontal detail image, a vertical detail image, and a diagonal detail image. The low-pass subimage represents the lowfrequency information in the image, which corresponds to smooth regions. The high-pass subimages represent the highfrequency information in the image, which correspond to detailed image information. Based on the characteristics of these subimages, the most effective method is selected to enhance the coefficients of the different frequency components. Finally, the enhanced image in the spatial domain is obtained through inverse transformation.
The steps of WT-based image enhancement are as follows [163], [164].
(i) The original image is input.
(ii) The low-frequency and high-frequency components of the original image are obtained via wavelet decomposition. VOLUME 8, 2020 (iii) The wavelet coefficients are nonlinearly enhanced with a functional relationship that satisfies where G is the gain for the wavelet coefficient, T is the threshold for the wavelet coefficient, W i is the wavelet coefficient after image decomposition, and W o is the wavelet coefficient after enhancement.
(iv) The enhanced wavelet coefficients are inversely transformed to obtain the reconstructed enhanced image.
The basic flow of the WT-based image enhancement process is illustrated in Fig. 20.  The results of WT-based image enhancement are shown in Fig. 21, where n is the wavelet scale.
In low-light images, it is difficult to distinguish image noise from image details. A high-frequency analysis is conducted on the WT-decomposed image, and then, to process the decomposed wavelet coefficients, various thresholds and enhancement factors are applied to effectively remove noise while enhancing detail components. Generally, the enhancement effect is better than that of traditional image enhancement algorithms [166]. Zong et al. proposed a contrast enhancement method based on multiscale wavelet analysis [167]. In this method, a multiscale nonlinear high-pass function is used to process the wavelet coefficients, thus enabling the enhancement of ultrasonic images. Loza et al.
proposed an adaptive contrast enhancement method based on the statistics of local wavelet coefficients [168]. A model for the local wavelet coefficients was established on the basis of the binary Cauchy distribution, thus yielding a nonlinear enhancement function for wavelet coefficient compression. A WT-based image enhancement algorithm based on a knee function and gamma correction (KGWT) has been proposed in which an improved knee function and a gamma transform function are used to enhance the low-frequency coefficients [169]. Then, after enhancement, the low-frequency coefficients are combined with the high-frequency coefficients, and finally, the inverse WT is applied to obtain the enhanced image. The KGWT algorithm improves the overall brightness and contrast of images. A WT-based image enhancement algorithm based on contrast entropy was proposed in [170]. After wavelet decomposition, the lowfrequency components of the image are enhanced via HE, and the high-frequency components are enhanced by maximizing the contrast entropy. Likewise, in [171], the singular value matrices of low-frequency images were obtained with an enhanced wavelet decomposition approach, which also achieved an improved image enhancement effect. A fast and adaptive enhancement algorithm for low-light images based on the WT was proposed in [172]. In this algorithm, the RGB input image is transformed into HSV space, and the discrete wavelet transform (DWT) is applied to the brightness (V ) image to separate high-frequency and low-frequency subbands. The illumination components in the low-frequency WT subbands of the image are rapidly estimated and removed using bilateral filtering, while a fuzzy transformation is used to realize the enhancement and denoising of edge and texture information. WT-based image enhancement theory is often combined with other approaches, such as fuzzy theory, image fusion, and HE. As discussed in [173], [174], after wavelet decomposition is performed on the original image, HE can be performed on each subband image individually. Finally, the inverse WT can be used to reconstruct the enhanced, noise-reduced image [175]. The WT approach has also been combined with Retinex theory to enhance low-light images, thus achieving a better enhancement effect [176], [177]. Russo [178] proposed a method of improving image quality by means of multiscale equalization in the wavelet domain. Chen [179] proposed an image enhancement method that combines wavelet and fractional differential models. The WT can reflect both the time-domain and frequency-domain features of an image. Specifically, this model not only extracts edge information from an image but also extracts its overall structure, which is consistent with the needs of low-light image enhancement. However, because the wavelet basis needs to be defined in advance, the application of this algorithm is limited.
The curvelet transform is a multiscale analysis method developed based on the WT that can overcome the limitations of the WT by enhancing the curved edges in an image [180]. Starck et al. [181] proposed a multiscale analysis method based on the curvelet transform and compared it with the WT algorithm to demonstrate its superiority for color image enhancement. The curvelet-transform-based enhancement algorithm achieves a better effect for noisy images; however, it is not as effective as the WT method for noiseless or nearly noiseless images. In [182], the WT was combined with the curvelet transform to achieve image enhancement with edge preservation. The specific steps are as follows. First, the curvelet transform is applied to remove noise without the loss of edge details; then, the image is enhanced using the WT. In [183], an improved enhancement algorithm for noisy lowlight color images based on the second-generation curvelet transform was proposed. A compromise factor for the YUV 87896 VOLUME 8, 2020 (luminance and chromaticity) color space and an improved gain function were used to suppress, elevate or maintain the curvelet coefficients. This approach effectively suppresses noise while optimally recovering both the edges and smooth parts in an image acquired under low illumination. Thus, the image enhancement quality is effectively improved. The main advantage of image enhancement methods based on the WT is that they allow the time-and frequency-domain features of images to be analyzed on multiple scales [184]. Another advantage of wavelet analysis lies in the refined local analysis capabilities, as such methods have better local characteristics in both the spatial and frequency domains and thus are beneficial for analyzing and highlighting the details of an image. Wavelet analysis is mainly used for infrared images [185], [186] and medical image enhancement [187]. The disadvantage is that overbright illumination cannot be avoided.
In summary, frequency-domain-based algorithms can effectively highlight the details of an image through enhancement of the wavelet coefficients, but they can also easily magnify the noise in the image. Like other frequency-domain transformation methods, these image enhancement methods require large amounts of calculation, and the selection of the transformation parameters often requires manual intervention.

E. METHODS BASED ON IMAGE FUSION
Another direction of research on low-light image enhancement involves methods based on image fusion techniques [188]. In these methods, many images of the same scene are obtained with different sensors, or additional images are obtained with the same sensor using various imaging methods or at different times. Finally, as much useful information as possible is extracted from every image to synthesize a high-quality image, thus improving the utilization rate of the image information. The synthesized image can reflect multilevel information from the original images to comprehensively describe the scene, thus allowing the available image information to better meet the requirements of both human observers and computer vision systems.

1) MULTISPECTRAL IMAGE FUSION
Multispectral image fusion is an improved method of obtaining the details of a low-light imaged scene by fusing a visible image with an infrared image. Near-infrared (NIR) light has a longer wavelength and stronger penetration ability than does visible light, allowing redundant information to be removed from a filtered infrared image. Additionally, a low-light visible image can provide rich background information; consequently, better images can be obtained through image fusion. For example, in a method developed by the US Naval Research Laboratory (NRL), images obtained with an infrared thermal imager are integrated with the R-, G-and B-channel images obtained with a low-light night vision device to obtain night vision color images [189]- [191]. Toet et al. proposed a pseudocolor fusion algorithm for infrared and visible images [192]; this algorithm enhances the clarity of the image details while retaining the unique information captured by various sensors. Additionally, this algorithm involves only addition and subtraction operations and thus can be implemented using simple hardware in real time [193]. Zhu et al. proposed a fusion framework for night vision applications called night vision context enhancement (FNCE) [194], in which the fused result is obtained by combining decomposed images using three different rules.
Furthermore, many scholars have studied the use of night vision technology for single-and double-channel low-light color fusion based on bispectral and trispectral features. Vision technology has been developed based on low-light and infrared thermal image fusion, low-light and longwave infrared image fusion, ultraviolet and low-light image fusion, and even trispectral color fusion based on low-light, mediumwave and longwave infrared images [195]- [197]. However, the visible and infrared images need to be acquired simultaneously, which constrains such algorithms in terms of the hardware conditions necessary to support them. Moreover, the intelligence and adaptability of these algorithms are poor, and their parameters need to be artificially set. Therefore, these algorithms have still not been widely adopted.

2) IMAGE FUSION BASED ON BACKGROUND HIGHLIGHTING
Generally, image fusion methods based on background highlighting rely on the integration of low-light images with daytime images to enhance the image details, thus improving the visual effect of the low-light images [198]. The general process is described as follows. First, an image is obtained in the daytime under reasonably sufficient lighting conditions for use as the source of the background for the fused image. Then, another image is obtained in the same position under low illumination, and the background of this image is removed. The remainder of the latter image is taken as the foreground of the fused image. Finally, the background and foreground are integrated into a single image using a suitable algorithm. For example, Raskar et al. estimated the intensity of the mixed gradient field of multiple low-light images and daytime images of the same scene, thus improving the visual effect of the low-light images [199]. Rao et al. proposed a low-light enhancement method based on video frame fusion [200]. The foreground area of each low-light video frame was fused with the background area from a daytime video frame of the same scene to improve the brightness of the low-light video and compensate for detail loss. In [201], daytime images from the same site at various times were fused, and the final fused image was obtained using a moving object extraction technique and weighting processing based on brightness estimation theory. This process is shown in Fig. 22.
Multi-image fusion methods such as those presented in [202], [203] achieve a better enhancement effect but require high-quality daytime video information from the same scene. For example, such methods are not suitable for use in underground mines, because no high-quality daytime video information is available for such areas. Therefore, the applications of these algorithms are limited. Moreover, the large number of iterations required complicates the calculation.

3) FUSION BASED ON MULTIPLE EXPOSURES
Image fusion is the process of combining multiple images of the same scene into a single high-quality image that contains more information than any single input image. Petschnigg et al. proposed a method of obtaining various images with both flash and nonflash technologies and then realizing low-light image enhancement through image fusion [204]. In this method, a flash image is captured to record detailed information of the scene, and a nonflash image is captured to record the brightness information of the background. Then, the image detail information is integrated with the background brightness information. The resulting image contains not only the details from the flash image but also the brightness information from the nonflash image. Similarly, high-dynamic-range (HDR) [205]- [207] imaging using multiexposure fusion (MEF) techniques has become very popular in recent years. MEF methods use multiple images of the same scene with different exposures. The final HDR image is obtained by synthesizing the best details from the images corresponding to each exposure time. A gradientdomain HDR compression algorithm was proposed in [208]. In this algorithm, different gradients are proportionally compressed in the gradient domain of the images, and Poisson's equation is solved in the modified gradient domain to obtain output images with a low dynamic range. This algorithm can also reveal detailed information in areas of various brightness in HDR night images. Li et al. proposed an image enhancement algorithm based on multiple image fusion [209]. In this algorithm, multiple images of the same scene are first acquired with different exposure times, and then, various details are extracted from each image. Finally, these details are integrated to generate an enhanced HDR image [210]. Merianos and Mitianoudis combined two image fusion methods, one for the fusion of the luminance channel and one for the fusion of the color channels. The fusion output thus derived outperforms both individual methods [211]. In Ref. [212], the author proposed a new fusion approach in the spatial domain using a propagated image filter. In the proposed approach, a weight map is calculated for every input image using the propagated image filter and gradient-domain postprocessing.
The fusion of multiple images acquired from the same scene can be applied to effectively enhance low-light images.
Because good-quality image information from the same scene is needed, methods of this kind have stringent requirements in terms of image acquisition; in particular, the camera equipment needs to be stable. Since a long shooting time is required, this method cannot be applied for real-time imaging or video enhancement. Moreover, the enhancement effect for images of globally low brightness is poor.

4) FUSION BASED ON A SINGLE IMAGE
Many scholars have studied the synthesis of the entire dynamic range of a scene [213], [214], including details extracted in a variety of ways from a single image, to break the dependence on image sequences, as shown in Fig. 23. Le and Li [215] improved the contrast of an image by fusing the original image with the image obtained after a logarithmic transformation. Yamakawa and Sugita presented an image fusion technique that used a source image and the Retinexprocessed image to achieve high visibility in both bright and dark areas [216]. In Ref. [217], Wang et al. adaptively generated two new images based on nonlinear functional transformations in accordance with the illumination-reflection model and multiscale theory and used a principal component analysis (PCA)-based fusion method to enhance a low-light image. In [218], an adaptive histogram separation method was used to construct underexposed and overexposed images from an original image sequence; these images were then separately processed, and finally, HDR images were generated via multiexposure image fusion. In addition, Fu et al. [219] proposed an image enhancement algorithm based on the fusion of the results of multiple enhancement techniques. This algorithm integrates multiple image enhancement techniques by means of a linear weighted fusion strategy to improve the enhancement effect. However, this strategy is too complex to satisfy real-time requirements. The algorithm proposed in [220] integrates the color contrast, saturation and exposure brightness of an original or preprocessed image by incorporating MSRCR into a pyramid algorithm using the gold tower technique and specifying different weight parameters depending on the image information to achieve the effective color enhancement of a traditional low-light image. A camera response model (CRM) is often adopted for generating multiple images [221]. In [222], the authors proposed a single-image-based method of generating HDR images based on camera response function (CRF) reconstruction. Ying et al. [223] proposed a novel bioinspired enhancement model in which the source image is generated on the basis of a simulated CRM, and the weight matrix for image fusion is designed using illumination estimation techniques [224], [225]. Unlike the model presented in [223], the model presented in the later cited papers avoids any heuristic judgment of whether an image pixel is underexposed and thus is more flexible in generating more intermediate enhancement results. In Ref. [226], a framework based on a CRM and a weighted least squares strategy was proposed in which every pixel is adjusted in accordance with a calculated exposure map and Retinex theory; this framework can preserve details while improving contrast, color correction, and noise suppression. In addition, Zhou et al. [227] generated multiple enhanced images based on a lightness-aware CRM and then performed mid-level fusion of these images based on a patch-based image decomposition model. This model, however, has a limited ability to improve images in which one area is already overenhanced. In this case, the overenhanced area is even more strongly enhanced, resulting in the loss of important details.
In short, the main idea of methods based on image fusion is that useful information on the same target collected from multiple sources can be further utilized, without requiring a physical model, to obtain a final high-quality image through image processing and computer technology.
These fusion-based methods are simple and can achieve good results. However, they require two or more different images of the same scene; therefore, it is difficult to realize image enhancement within a short time, as is needed for realtime monitoring situations, and these methods are difficult to apply and popularize in practice.

F. METHODS BASED ON DEFOGGING MODELS
As one branch of the field of image enhancement, image defogging techniques have seen great progress and produced good results in recent years. In 2009, He Kaiming proposed the dark channel prior theory for images, which has been widely applied [228]. In 2011, a low-light enhancement algorithm [229], also called a bright channel prior method [230], [231], was proposed by Dong et al. based on defogging theory; this method relies on the statistical analysis of a dark primary color version of a low-light inverted image and a dark primary color version of a foggy image. The main idea of the algorithm is that when an RGB image captured in a dark environment is inverted, the visual effect is similar to that of a daytime image acquired in a foggy environment (as shown in Fig. 24). Hence, a defogging algorithm based on a dark channel prior can be used to process the inverted lowlight image; then, the image can be inverted again to obtain an enhanced low-light image.
This enhancement method greatly improves the image brightness and enhances the visual details of the image by analyzing the features of the low-light image and modeling them with a foggy image degradation model. The basic process is shown in Fig. 25. Low-light image enhancement methods based on dark primary color defogging techniques  have certain issues. For example, low-light images inevitably contain noise; however, the foggy image degradation model used in such an algorithm does not consider the effect of noise. Therefore, the image noise will typically be amplified, which will visually impact the results of image enhancement [232]- [234]. Considering the need for noise processing, Liu Yang et al. optimized the processing speed of such an algorithm without accurately extracting the transmittance; therefore, the final enhanced images exhibited a blocky effect. Zhang et al. also proposed an optimized algorithm, in which the parameters for transmittance estimation are selected directly based on experience; consequently, the robustness of this algorithm is poor [235]. Jiang et al. used filters to remove details and introduced a pyramid operation to calculate a smooth transmission coefficient, which not only improved the processing speed but also yielded better naturalness [236]. Simultaneously, the noise was suppressed. Later, Song et al. [237] improved upon this model to overcome an issue related to block artifacts. Then, Pang introduced a gamma transformation to improve the image contrast [238]. By combining the defogging approach with bilateral filtering, Zhang et al. proposed a low-light image enhancement method that can operate in real time. After the VOLUME 8, 2020 parameters are initially estimated using a dark channel prior, they are optimized using a bilateral filter; thus, the effect of noise is reduced [239]. Tao et al. combined a bright channel prior with a convolutional neural network (CNN) [240], and Park et al. combined a bright channel prior [241] with a Retinex enhancement algorithm. Both achieved improved results [242]. A fast enhancement algorithm for low-light video has been proposed by combining Retinex theory with dark channel prior theory [243], and this algorithm can be further combined with scene detection, edge compensation and interframe compensation techniques for video enhancement. In [244], a method was proposed to solve the transmittance problem based on a foggy degradation model and a CNN, in which the transmission map and atmospheric light map are amended by means of guided filtering to obtain an enhanced low-light image. Recently, an enhancement method with strong illumination mitigation and bright halo suppression has been presented, which combines a dehazing algorithm with a dark channel prior and a denoising method to achieve a better visual effect [245].
Algorithms based on defogging models offer good performance with low computational complexity. However, their physical interpretation is somewhat lacking, and they are still susceptible to overenhancement in some detailed areas. Inverted low-light images have their own unique characteristics, and the direct application of defogging algorithms to such images is still not an ideal approach for image enhancement.

G. METHODS BASED ON MACHINE LEARNING
Most existing low-light image enhancement techniques are model-based techniques rather than data-driven techniques. Only in recent years have methods based on machine learning for image enhancement begun to emerge in significant numbers [246]- [248]. For example, in [249], the reflection component of an object was represented using a sparse representation method, and the image details contained in the reflection component were learned using a dictionary learning method, thus achieving an improved enhancement effect. However, noise can easily be introduced during the machine learning process. An image enhancement method based on a color estimation model (CEM) was proposed by Fu et al. [250], in which the dynamic range of color images in the RGB color space was controlled by adjusting the CEM parameters to effectively inhibit oversaturation of the enhanced images. Fotiadou et al. proposed a low-light image enhancement algorithm based on a sparse image representation [251] in which both a low-light condition dictionary and a daylight condition dictionary were established. The sparse constraint was used as prior knowledge to update the dictionaries, and low-light image blocks were used to approximately estimate the corresponding daylight images. An image enhancement algorithm based on fuzzy rule reasoning [252] was proposed in which three traditional enhancement methods were combined by applying fuzzy theory and machine learning to establish a set of fuzzy rules, and the best enhancement algorithm was adaptively selected for different images to achieve image enhancement. This method can also be used to objectively and accurately evaluate the image enhancement effect.
Since 2016, several deep-learning-based methods for image enhancement have also emerged. For example, Yan et al. proposed the first deep-learning-based method for photo adjustment [253]. Lore et al. adopted a stacked sparse denoising autoencoder in a framework for training an LLNet for low-light image enhancement [254]. In this framework, a sparsity regularized reconstruction loss was taken as the loss function, and deep learning based on the self-encoder approach was used to learn the features of image signals acquired under various low-illumination conditions to realize adaptive brightness adjustment and denoising. Park et al. proposed a dual autoencoder network model based on Retinex theory [255]; in this model, a stacked autoencoder was combined with a convolutional autoencoder to realize low-light enhancement and noise reduction. The stacked autoencoder, with a small number of hidden units, was used to estimate the smooth illumination component in the space, and the convolutional autoencoder was used to process two-dimensional image information to reduce the amplification of noise during the process of brightness enhancement.
CNNs have been used as the basis of deep learning frameworks in many research works [256]- [259]. Tao et al. proposed a low-light CNN (LLCNN) in which a multistage characteristic map was used to generate an enhanced image by learning from low-light images with different nuclei [260]. In [261], a global illumination-aware and detail-preserving network (GLADNet) was designed. In this network, the input image is first scaled to a certain size and then passed to encoder and decoder networks to generate global prior knowledge of the illumination. Based on this prior information and the original images, a convolutional network is then used to reconstruct the image details. Ignatov et al. took a different approach of learning a mapping between images acquired by a mobile phone camera and a digital single-lens reflex (DSLR) camera. They built a dataset consisting of images of the same scene taken by the different cameras [262] and presented an end-to-end deep learning approach for translating ordinary photos into DSLR-quality images. Lv et al. proposed a new network (MBLLEN) consisting of a feature extraction module (FEM), an enhancement module (EM) and a fusion module (FM) [263], which produces output images via feature fusion. Gabriel et al. designed a deep convolutional neural network (DCNN) [264] based on a large dataset of HDR images, and Liu et al. trained the DCNN using only synthetic data to recover the details lost due to quantization [265]. Gharbi et al. constructed a learning framework based on a deep bilateral network, thus achieving real-time processing for image enhancement [266]. Later, Chen et al. [267] introduced a dataset of raw short-exposure low-light images (the See-in-the-Dark (SID) database) and developed a pipeline for processing these images based on a fully convolutional network (FCN). Through end-to-end training, a good improvement over the traditional method of low-light image processing was achieved.
Based on Retinex theory, Shen et al. [268] analyzed the performance of the MSR algorithm from the perspective of a CNN framework and proposed a method of enhancing lowlight images by using an MSR network (MSR-net) based on a CNN architecture, while Guo et al. [269] proposed our pipeline neural network consisting of a denoising net and low light image enhancement net (LLIE-net). Wei et al. assumed that observed images could be decomposed into their reflectance and illumination components, collected a LOw-Light (LOL) dataset containing low-/normal-light image pairs, and proposed a deep network called Retinex-Net [270]. Li et al. designed a network called LightenNet based on a CNN architecture [271]; this network takes a weakly illuminated image as input and outputs a corresponding illumination map, which is subsequently used to generate an enhanced image based on the Retinex model. Zhang et al. built a simple yet effective network called the Kindling the Darkness (KinD) network [272], which is composed of a layer decomposition net, a reflectance restoration net, and an illumination adjustment net, and trained it on pairs of images captured under different exposure conditions. Inspired by the multiple image fusion method, Cai et al. [273] proposed a framework based on a CNN trained to enhance single images. In this work, thirteen different MEF and HDR compression methods were used to generate an enhanced image for each series of images from a large-scale multiexposure image dataset. Finally, low-light images were enhanced by the CNN after end-to-end training on the lowcontrast and high-contrast image dataset. Yang et al. used two CNNs to build a tool for RGB image enhancement [274] in which intermediate HDR images are first generated from the input RGB images to ultimately produce high-quality LDR images. Nevertheless, generating HDR images from single images is a challenging problem. To handle both local and global features, Kinoshita and Kiya proposed an architecture consisting of a local encoder, a global encoder, and a decoder trained on tone-mapped images obtained from existing HDR images [275]. Experimental results showed its excellent performance compared with conventional image enhancement methods, including CNN-based methods.
In contrast to supervised learning methods, generative adversarial network (GAN)-based methods can be used for image enhancement without training on pairs of images [276], [277]. For example, Meng et al. proposed a GAN-based framework for nighttime image enhancement, which takes advantage of GANs' powerful ability to generate images from real data distributions, and the results demonstrate its effectiveness. To our knowledge, this was the first time that GANs were applied for the purpose of nighttime image enhancement [278]. In Ref. [279], the authors fully utilized the advantages of both GANs and CNNs, using a transitive CNN to map the enhanced images back to the space of the source images to relax the need for paired groundtruth photos. Kim et al. [280] applied local illumination to make the training images and used an advanced generative adversarial network to build Low-Lightgan. The key advantages of such networks are that they can be trained easily and can achieve better experimental results than traditional enhancers [279].
Undoubtedly, deep-learning-based methods can achieve excellent performance in low-light image enhancement, and they also represent a major trend of current development in image processing research. However, such methods must be supported by large datasets, and an increase in the complexity of a model will lead to a sharp increase in the time complexity of the corresponding algorithm. With the steady growth of research on low-light image enhancement, not only are some low-light data available from widely used public benchmark datasets such as PASCAL VOC [281], ImageNet [282], and Microsoft COCO [283], but researchers are also building public datasets specifically designed for low-light image processing, such as SID [267] and EDD (Exclusively Dark Dataset) [284].

III. EVALUATION METHODS
Image quality assessment (IQA) focuses mainly on two aspects, namely, the fidelity of the image and the readability of the image, which can be regarded as subjective and objective evaluation standards, respectively. A subjective evaluation method measures image quality on the basis of the subjective perception of the human visual system, i.e., whether the image conveys a certain experience. However, it is still difficult to accurately simulate the human visual system. Therefore, current subjective evaluation systems based on the human visual system can evaluate image quality only qualitatively rather than quantitatively [285].

A. SUBJECTIVE EVALUATION
In a subjective evaluation method, human observers are asked to evaluate the quality of processed images in accordance with their visual effects based on a predetermined evaluation scale. Such an evaluation depends on subjective assessment of the image processing results to determine the advantages and disadvantages of a particular algorithm. The score is typically divided into 5 grades (1-5 points), and the number of raters should typically be no fewer than 20 [286]. Some of the raters should have experience in image processing, while some should not. The raters will evaluate the visual effects of the images in accordance with their personal experience or agreed-upon evaluation criteria. To ensure fairness and equity, the final scores will be weighted to obtain the final subjective quality evaluation result for each image. The typical evaluation standards are summarized in Table 1.
This method is simple and can reflect the visual quality of images. Such a subjective evaluation can accurately represent the visual perception of the majority of observers. However, such an evaluation lacks stability and can be easily affected by the experimental conditions as well as the knowledge background, emotional state, motivation and degree of fatigue of the observer. In studies related to image enhancement, it is necessary to provide key details of different magnified parts of images for comparison to assess, e.g., lack of uniformity. However, this process is time consuming and arduous in practice and thus often cannot be applied in engineering applications.

B. OBJECTIVE EVALUATION
An objective evaluation is an evaluation using specific data and based on certain objective criteria. To the best of our knowledge, there are no IQA methods that have been specifically designed for the evaluation of low-light image enhancement methods. Hence, different researchers utilize different strategies to evaluate their results. At present, the objective evaluation methods for image enhancement can be divided into full-reference methods and no-reference methods depending on whether they require reference images (ground-truth images or synthetic images). Objective evaluation methods have various advantages, such as simple calculations, fast execution, ease of quantitative calculation based on a constructed model, and high stability; therefore, data from objective evaluations are generally adopted as image quality scores [287].

1) NO-REFERENCE IQA (NIQA) METRICS
Since no objective reference image is available in the case of a low-light input image, most methods that are suitable for low-light image enhancement assessment are based on NIQA metrics. The most common NIQA metrics include the mean value (MV), standard difference (STD), average gradient (AG), and information entropy (IE). In addition, there are several general methods available for image quality evaluation, including the Blind/Referenceless Image Spatial QUality Evaluator (BRISQUE) [288], the Naturalness Image Quality Evaluator (NIQE) [289], the BLind Image Integrity Notator using DCT Statistics (BLIINDS-II) [290], the blind tonemapped image quality index (BTMQI) [291], gradient ratioing at visible edges (GRVE) [292], the autoregressive-based image sharpness metric (ARISM) [293], the no-reference image quality metric for contrast distortion (NIQMC) [294], the Global Contrast Factor (GCF) [295], the average information content (AIC) [296], the effective measure of enhancement (EME) [297], PixDist [298], and the no-reference free-energy-based robust metric (NFERM) [299]. The commonly used NIQA metrics and related references are shown in Table 2. Descriptions of several of these metrics follow. (i) Mean value (MV). The MV mainly refers to the mean of the gray values of an image, and it mainly reflects the color or degree of brightness of the image. The smaller the image mean is, the darker the image. Conversely, the larger the mean is, the brighter the image, and the lighter the colors. The formula is as follows: where M and N are the width and height, respectively, of the image and f (i, j) is the gray value at pixel point (i, j).
(ii) Standard difference (STD). The variance of the gray values reflects the degree of dispersion of the image relative to the mean and thus is a measure of the contrast within a certain range. The larger the variance is, the more information is contained in the image, and the better the visual effect. When the variance is smaller, the information contained in the image is less, and the image is more monochromatic and uniform. The formula is where M and N are the image width and height, respectively; f (i, j) is the gray value at pixel point (i, j); and µ is the MV of the image (iii) Average gradient (AG). The AG represents the clarity of an image, reflecting the image's ability to express contrasting details. This metric measures the rate of change in the image values based on changes in the contrast of minute details or the relative clarity of the image. In an image, faster gray changes in a certain direction result in larger image gradients; therefore, this metric can be used to determine whether or not an image is clear. The AG can be expressed as where M and N are the image width and height, respectively, and ∂f /∂x and ∂f /∂x are the horizontal and vertical gradients, respectively. (iv) Information entropy (IE). Entropy can be used as a measure of an amount of information and is widely used to evaluate image quality [300,48]. A static image is regarded as an information source with random output; the set A of source symbols is defined as the set of all possible symbols {a i }, and the probability of source symbol a i is P(a i ). Thus, the average information quantity of an image can be expressed as According to entropy theory, the larger the IE value is, the larger the amount of information contained in the image, and the richer the image detail.
(i) Mean square error (MSE). This metric represents the direct deviation between the enhanced image and the original image; it has the same meaning as the absolute mean brightness error (AMBE) [75].
where M and N are the width and height, respectively, of the image; f (i, j) represents the input image; and f e (i, j) represents the enhanced image. In an image quality evaluation, a smaller MSE value indicates higher similarity between the enhanced and original images.
(ii) Peak signal-to-noise ratio (PSNR). The PSNR of an image is the most extensively and commonly used objective evaluation method for measuring the image denoising effect. The larger the PSNR value is, the smaller the difference between the images before and after processing. An excessively high PSNR indicates that the effect of the denoising algorithm is not obvious. A smaller PSNR indicates a greater difference between the images before and after processing. An excessively low PSNR may suggest that the image is distorted. The specific expression is as follows: where f max is the maximum gray value, f max = 255.
(iii) Structural similarity index metric (SSIM). The above methods do not consider the characteristics of the human visual system when assessing image quality; they compute only a simple random error between the input image and the processed image and analyze the difference between the input and output images from a mathematical perspective. Therefore, the above metrics cannot fully and accurately reflect the image quality. Researchers have found that natural images exhibit certain special structural features, such as strong correlations between pixels, and these correlations capture a large amount of important structural information for an image. Therefore, Wang et al. proposed a method based on structural similarity for evaluating image quality [301]. The SSIM evaluates the quality of a processed image relative to the reference image based on comparisons of luminance (l(f, f e )), contrast (c(f, f e )) and structure (s(f, f e )) between the two images. These three values are combined to obtain the overall similarity measure. The formula is as follows: The flow of the SSIM algorithm is shown in Fig. 26. The degree of similarity between the two images is reflected by the value of the SSIM; the minimum value is 0, and the maximum value is 1. A value closer to 1 indicates that the two images are more similar. Taking the human visual system as the starting point, this method can effectively simulate human visual perception to extract information about the structure of an image. The evaluation result is very close to the subjective perception of the human eye; therefore, this metric is widely used in image quality evaluations.
(iv) Lightness order error (LOE). Considering that the relative order of lightness of different image areas reflects the direction of the light source and the variation in illumination, Ref. [136] proposed the LOE metric to measure the discrepancy in lightness order between an original image I and its enhanced version I e . The LOE is defined as where RD(i, j) is the difference in the relative lightness order between the original image f and its enhanced version f e at pixel (i, j). This difference is defined as follows: where M and N are the image width and height, respectively; ⊕ is the exclusive-or operator; and L(i, j) and L e (i, j) are the maximum values among the three color channels at location (i, j) for f and f e , respectively. The function U (p, q) returns a value of 1 if p >= q; otherwise, it returns 0. The smaller the LOE value is, the better the lightness order is preserved. The above measures have the following advantages: they are simple to calculate, have clear physical meanings, and enable mathematically convenient optimizations.

IV. ANALYSIS OF DIFFERENT ENHANCEMENT METHODS
To compare the enhancement effects of various algorithms as well as the consistency of subjective and objective evaluation, experiments using many methods are presented in this paper for illustration. A test platform was built based on a desktop computer to verify the algorithms. This system includes an Intel(R) Core(TM) i7-6700 CPU @3.4 GHz with 16 GB RAM and the Windows 10 operating system.

A. SUBJECTIVE EVALUATION
The test images shown in Fig. 27 represent three different illumination conditions, namely, uniform low light, uneven illumination and nighttime; the source images are named 'Flowers.bmp', 'Building.bmp' and 'Lawn.bmp', respectively. In addition, we adopt two pairs of images for referencebased comparisons, where each pair consists of a low-light image, as shown in the top row of Fig. 28, and a corresponding well-exposed image, as shown in the bottom row. In Fig. 28, the image on the left is named 'Desk.bmp', and the image on the right is 'Road.bmp'.
The experimental results are shown in Figs. 29-33. In these figures, panel (a) contains the original image, and panels (b)-(r) display the results of many enhancement methods: Gamma correction [3], AHE [78], WT [166], BIMEF [223], CegaHE [102], CRM [225], CVC [89], Dong et al. [229], MBLLEN [263], HMF [101], LIME [13], MF [219], HE [66], MSRCP [127], MSRCR [115], NPE [136], and SRIE [126]. As shown in panels (b)-(r), all of these image enhancement methods improve the visual effect of the original image to some degree. The details become clearer with the Gamma, WT, AHE, HMF, CVC, MSRCP and SRIE methods, but the overall level of brightness is dark. Especially when the Gamma and CVC methods are used, the enhancement effects for the three types of images are similar, while the WT method makes the output image blurred. The AHE method achieves a better effect when processing uniformly illuminated lowlight images, but a wheel halo effect appears in unevenly illuminated low-light images. Although the CegaHE, HE, and MSRCR methods can brighten the entire image, the hue changes dramatically for an image with uneven illumination, resulting in the loss of the real color of the original scene. Although the image brightness after processing with the SRIE method is not high, this method achieves a consistent image processing effect for all three types of images, and the tone recovery effect is superior. By comparison, the Dong, MBLLEN, MF, NPE, LIME and CRM methods demonstrate  outstanding performance in color and detail enhancement, and their visual effects are obviously superior to those of the other abovementioned image enhancement methods. However, when the Dong and NPE methods are used to process unevenly illuminated low-light images such as the 'Building' and 'Road' images, overenhancement appears at the boundaries. The MF method, MBLLEN method and CRM method better maintain the color of the original images compared with the above methods, but their overall effect is no better than that of the LIME method. The LIME method considers both brightness and hue information and maintains excellent realistic effects. Hence, the LIME method achieves higher color fidelity from the perspective of human visual perception.

B. OBJECTIVE EVALUATION
Based on the above images, experiments for objective quality evaluation were performed using various IQA methods, including both NIQA and FIQA metrics.

1) NIQA-BASED EVALUATION
Eight metrics, namely, STD, IE, AG, BLIINDS-II [290], NIQE [289], BRISQUE [288], the contrast enhancementbased contrast-changed image quality measure (CEIQ) [311], and the spatial-spectral entropy-based quality measure (SSEQ) [312], were employed for NIQA-based evaluation. The experimental results obtained on 'Flowers.bmp', 'Building.bmp' and 'Lawn.bmp' are shown in Tables 4-6, and the best score in terms of each metric is highlighted in bold. VOLUME 8, 2020  These data show that the different evaluation metrics assign different scores to the same image enhancement algorithm and that the interpretations of the evaluation results are completely opposite in some cases. The reason is that the traditional IQA metrics used in this evaluation consider different aspects of the image obtained after enhancement. For the three images, no method gets the best score on all metrics. The HE method achieves the highest scores in terms of the IE and CEIQ metrics on the three images. The LIME method achieves the best scores in terms of the AG metric for 'Building.bmp' and 'Lawn.bmp'. The CVC method achieves the best score on STD metric for ''Flower.bmp'' and 'Lawn.bmp'. Overall, the HE and CVC methods are the top two scoring methods based on these metrics. However, to some extent, the distortion of chrominance information causes the results of the objective evaluation to be opposite to those of the subjective evaluation.
For example, the CegaHE method earns the best scores according to seven of the above eleven metrics on 'Desk.bmp'. For the image 'Road.bmp', BIMEF, CRM and VOLUME 8, 2020   MF achieve four, four and two of the best scores, respectively. In addition, these three methods have the same scores in terms of the GMSD and VSI metrics for the evaluation on 'Road.bmp'. To some extent, FIQA-based evaluations provide a more accurate description of the images and are more consistent with subjective evaluation results than NIQA-based evaluations are for cases in which reference images are available.

C. TIME COMPLEXITY
To test the processing speeds of the various methods, experiments were performed using images of various sizes, and all algorithms were run using MATLAB except the MELLEN method [263]. Table 9 shows that the Retinex-based methods (MSR, MSRCR, and MSRCP) have high computational complexities because of their multiscale Gaussian filtering operations. The NPE, SRIE and MELLEN methods have the lowest computational efficiency for processing a single image because of the use of iterative computations to find the optimal solution. When processing an image with pixel dimensions of 3200 × 2400, their processing times are 206 seconds, 649 seconds and 890 seconds, respectively. In contrast, gamma correction and the various HE-based methods (AHE, HMF, and CegaHE) are faster, and their run times are only slightly affected by an increase in the image size. In particular, when the gamma correction method is run on an image of 3200 × 2400 pixels, it needs only 70 milliseconds to run, which is 1/12700th of the run time of the SRIE method. Therefore, the gamma correction method has an absolute advantage in terms of run time. For images with pixel dimensions of 1600 × 1200, the gamma method methods and the HE-based methods can be used under realtime conditions.
The IQA metrics considered above are not completely consistent with subjective human perception and thus are not suitable for the direct evaluation of enhanced low-light images. They need to be combined with subjective evaluations based on human vision. Therefore, there is a great need to design and develop an objective quality assessment method for low-light image enhancement that shows good agreement with the mechanism of human vision.

V. CONCLUSION
This paper summarizes seven widely used classes of low-light image enhancement algorithms and their improved versions and describes the underlying principles of the different methods. Then, it introduces the current quality evaluation system for low-light images and identifies the problems with this existing system. Finally, many representative image enhancement methods are evaluated using both subjective and objective evaluation methods. The characteristics and performance of the existing methods are analyzed and summarized, and the shortcomings of the present work in this field are further revealed. The essential purpose of low-light image enhancement is to improve the image contrast both globally and locally in a certain range of the gray space in accordance with the distribution of the gray values of the original image pixels. Simultaneously, it should be ensured that the enhanced image shows good image quality with regard to the characteristics of human visual perception, noise suppression, image entropy maximization, brightness maintenance, etc. The merits and shortcomings of the various methods are summarized in Table 10.
Based on the limitations of the current methods, care must be taken in image enhancement to ensure an appropriate balance among several factors, such as the image color, visual effect and information entropy, while attempting to improve the visibility of the image contrast. However, the existing algorithms all have certain disadvantages, such as loss of detail, color distortion, or high computational complexity; thus, current low-light image enhancement techniques cannot guarantee the performance of a vision system in a lowlight environment. In future research on low-illumination image enhancement, researchers should focus on the following tasks: (i) Improve the robustness and adaptive capabilities of low-light image enhancement algorithms. The robustness and adaptive capabilities of the existing methods are insufficient to meet the requirements of practical applications. The ideal method should be able to adaptively adjust to different application conditions and different types of low-light images.
(ii) Reduce the computational complexity of the available algorithms. To satisfy the needs of practical applications, realtime methods are often in demand; however, most of the existing methods currently require a long processing time. In addition, the results of the existing methods are still susceptible to certain problems, such as color deviations and detail ambiguity. The high-performance processors in graphics processing units (GPUs) allow such algorithms to be parallelized, which can significantly improve their processing speed and may ultimately enable real-time image enhancement.
(iii) Establish a standard quality evaluation system. At present, there are too few specialized low-light image datasets, and the quality evaluation system is not mature. This limits the further development of this research field and the selection of suitable enhancement and restoration methods for practical applications.
(iv) Develop a video-based enhancement algorithm. Currently, most of the research in this field has focused on single images, and research on video enhancement has not received sufficient attention; by contrast, video processing plays a greater role in practical applications. There is an urgent need to solve the problems related to the efficiency of low-illumination video processing, interframe consistency and so on.
In summary, thus far, no image enhancement algorithm exists that is optimal in terms of all of the above issues simultaneously. Therefore, it is necessary to select the most suitable image enhancement algorithm based on application-specific requirements. It is hoped that image enhancement technology can be advanced to a higher level through in-depth studies of these enhancement algorithms, thus allowing this technology to play an important role in multiple disciplines. Naturalness image quality evaluator Team of Shandong Provincial Education Department on robot's vision perception and control. He has participated in more than ten scientific research projects. He has published more than 60 articles on academic journals and conferences. He holds over 14 patents. His main research interests include computer vision, pattern recognition, and intelligent computing.

Abbreviation Phrase
XIAOJIN WU received the Ph.D. degree in traffic information engineering control from Beijing Jiaotong University, in 2011. He is currently with the College of Information and Control Engineering, Weifang University. He is also a Visiting Scholar with the University of North Texas, engaged in the research of image enhancement technology. He has published and authored more than ten articles on academic journals and conferences. His main research interests include intelligent systems and image processing ZAIRUI GAO received the Ph.D. degree in control theory and control engineering from the Ocean University of China, in 2012. Since 2012, he has been a Lecturer with the College of Information and Control Engineering, Weifang University, China. He has published and authored more than 20 articles on academic journals and conferences. His research interests include variable structure control, intelligent control, singular systems, and information processing. VOLUME 8, 2020