A Critical Analysis on Perceptual Contrast and Its Use in Visual Information Analysis and Processing

Contrast in visible images is one of the most relevant characteristics of visual signals. Since the pioneering works performed in vision psychology and optics, different definitions have been proposed in the literature. However, for the time being there exist no definition of contrast on which the vision research and visual processing scientific community can agree on. This makes it critical to have a clear view on the notion of contrast and its use in various applications. One issue to consider is how to define and particularly use this important measure in developing image processing and analysis methods. In this article, we present a critical review of contrast measures and associated models developed by the scientific community in vision, optics, and image processing. We also provide learned lessons and guidelines on the use of appropriate contrast measures in selected visual information processing and analysis applications. We discuss challenges and propose research avenues for models enriched by recent findings in the field of human vision research and machine learning. We believe that this work serves as a guideline and can potentially open new research perspectives to the scientific community working on visual information processing and analysis.


I. INTRODUCTION
Over the last sixty years, major advances have been seen in the field of image processing with a special focus on its application in various high-tech areas. This includes but is not limited to medical and scientific imaging, digital cinema, computational photography, biometrics, and remote sensing. With the development of new imaging modalities and multimedia products, many new approaches have been proposed in this field of research. Nevertheless, in recent years, research has seen a steady shift from understanding the image signal as a physical quantity and its interaction with the observer The associate editor coordinating the review of this manuscript and approving it for publication was Eduardo Rosa-Molinar .
to developing new mathematical models. In other words, we can see that omitting the human observer is the price paid for improving the research outcome in the field of image processing. Whereas, in many applications, such as diagnosis, recognition, and evaluation, through visual assessment of images, the human observer plays a prominent role in decision-making. Therefore, when it comes to designing multimedia processing techniques, exploiting knowledge about the Human Visual System (HVS) is a promising direction to take.
When it comes to the design of low-level and high-level tasks in image/video processing and computer vision, considering the perceptual aspects of the human observer and in some cases insects and animals have been an attractive VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ direction to take [1], [2]. Perceptual contrast is one of the most relevant psycho-physical factors that drives human vision.
Since the notion of contrast is mainly related to the human visual perception, in this work we interchangeably use the two term ''perceptual contrast'' and ''visual contrast'' to refer to the contrast as a psycho-physical quantity that conveys information about the way our HVS interacts with visual signal stimuli. When it comes to processing or analyzing optical signals or digital images, contrast is also a physical quantity that could be measured from the captured optical signal or numerical representation of the visual signal and used in machine-based applications. This is mainly the case of digital images and other digital representations of physical signals in various imagery modalities such as medical imagery, radar imagery, and hyperspectral imagery, to name a few. Indeed, contrast, i.e. the perception of stimuli in a given background, is a fundamental aspect of the human vision [3]. However, despite the continuing progress made by the vision research community, through several theoretical and experimental studies on contrast, the exploitation of the developed models and experimental results in image and video processing is still ongoing. While this topic of research has been studied and used in a wide range of applications such as Contrast Enhancement (CE) and visual signal coding, no universal definition of image contrast is agreed on. In fact, keeping in mind that the psycho-physical functioning of the HVS is not well understood, the concept of perceptual contrast is mainly subjective and could not be represented in an agreed universal and acceptable model. In this study, the lack of an agreed universal model has led to limit ourselves to few simple and well-established criteria and characteristics of the HVS. That is, the notion of luminance adaptation, color sensitivity, contrast sensitivity, i.e. frequency and directional selectivity and multi-scale/multi-resolution aspects that are also related to the visual cortex model [4]. The primary principle guiding this article is to understand the notion of contrast from its origin to the very recent development in the field of visual information processing and analysis. It is also a question of taking an enlightened look at the various works, both theoretical and experimental, in which the notion of contrast plays an important role, and of giving our point of view.
The importance of perceptual contrast is not only limited to the psycho-physical study of vision and modeling of the mechanisms of the HVS, but also its use in many disciplines related to image processing and image understanding. Contrast plays a prominent role in important applications such as medical imaging [5], Image Quality Assessment (IQA) and enhancement [6]- [8], visual signal analysis and coding [9], and other applications such as biometrics [10], rendering [11], and display quality evaluation [12]. It is worth pointing out that contrast is one of the first and widely investigated psycho-visual aspects in vision research [3], [13], [14].

B. CHALLENGES IN DEFINING PERCEPTUAL CONTRAST
An important issue to consider is how a contrast measure is defined and used in developing different image processing methods and analysis. The development of mathematical tools for signal processing and analysis has opened a formidable possibility of exploiting contrast in the design of efficient methods for processing and analyzing digital visual signals. The main goal of this article is to provide a critical study on the use of contrast in various perceptually inspired image processing applications. The study in [15] shows that using very simple models, it is possible to perform low-level processing such as noise filtering, edge detection, CE, and other complex applications including perceptual watermarking, perceptual quantization and coding, image fusion, and image segmentation. This article can be seen as a contribution to understanding the notion of contrast and its measurement, reviving perceptual approaches for low-level image processing and computer vision [1], [2], [15], [16], and opening new perspectives for solving various challenging problems. In summary, in this work we • present a critical review of contrast measures and associated models developed by the scientific community in vision, optics, and image processing, • provide learned lessons and guidelines on the use of appropriate contrast measures in selected visual information processing and analysis applications, • discuss challenges and propose research avenues for models enriched by recent findings in the field of human vision research and machine learning. The rest of the paper is organized as follows: we will first discuss historical descriptions of contrast through models and experiments in Section II followed by a discussion on digital contrast measures in Sections III. Section IV provides a summary of different contrast measures and the lessons learned from the mentioned measures. In Section V, we discuss in detail how different contrast measures can be used in various image processing and analysis applications. Finally, we conclude the paper in Section VI with future perspectives and challenges.

II. BASIC NOTIONS AND MODELS OF CONTRAST
The notion of contrast as defined and introduced in the early pioneering work of Bouguer [17], [18], Weber [19], Fechner [20], Hecht [21], and other theoretical and experimental work carried out until recently, allows us to draw a certain number of criteria and properties that any measure of contrast must satisfy. One of the first characteristics of contrast is to provide a measure of the relative change in the luminance of a stimulus observed in a background. An intuitive and empirical way to quantify this purely psycho-physical and sensory sensation is to form a relationship between the luminance of the target and that of its background.
The simplest yet intuitive measure is the relative luminance difference between the two. Other physical parameters affect this sensation and can be integrated in a measure referred to as visual contrast. It is clear that this measure is sensitive to any change of ambient luminance. Moreover, it depends on the frequency content of the observed scene. This is modeled through the Contrast Sensitivity Function (CSF) [22]. Contrast is therefore not invariant to the change in observation scale. The directional selectivity of the cortex well established by the two Nobel Prize laureates Hubel and Wiesel [4] also emphasizes that contrast varies with the direction of the stimulus. It is therefore quite natural for contrast measure to be variant to rotations. The color aspect and other high level factors are also important in a contrast measure. All these properties and characteristics of the HVS are to be considered when modelling visual contrast. In this section, we introduce and discuss the basic concepts, models, and experiments of historical optical contrast measures and particularly the Fechner [20], Michelson [23], and Moon and Spencer [24] contrasts. Here we refer to optical contrast as the measure which is defined through optical experiments where some continuous physical signals related to photometric quantities (e.g. luminance and intensity) and geometric parameters are involved. In the following we start by discussing the different historical contrast measures and highlight their strengths and shortcomings. Table 1 provides the description of the symbols and terms used in this article.

A. WEBER-FECHNER CONTRAST
The pioneering work by Weber in which he defined and measured the visual contrast [19] was later developed in a clear framework by Fechner one of Weber's students. The contrast as defined through the Fechner experiments and models are now known as the Weber-Fechner contrast [20]. However, this definition only applies to simple situations in which a uniform luminance background L contains an object with an incremental luminance L ( Fig. 1(a)). The aim of Weber-Fechner contrast is to determine the value of L, referred to as the Just Noticeable Difference (JND), which makes the object (target) just visible. In other words, using the Weber-Fechner contrast we can find the visibility threshold. To study the relationship between the increment ( L = L O − L) needed to make the object just perceptually noticeable ( Fig. 1), the ratio which is referred to as the Weber-Fechner (W-F) contrast is plotted as a function of the luminance value of the uniform background (L). Three sections can be observed from the log-log plot ( Fig. 1(b)) of the variation of the luminance increment ( L) against the luminance value of the background (L). First, the De Vries Rose section with a slope of 0.5 followed by the well-known Weber-Fechner section with a slope of one, and then the saturation zone with a slope of two. Note that this measure could be used in image processing in the condition that some assumptions and adaptations are provided. One of the simple adaptations is to assume that the pixel to be processed could be considered as the object and its neighbors as the background. Another adaptation is to consider an analysis window centered on the pixel to be processed as the stimulus and all the surrounding pixels as the background. In general, only the Weber-Fechner section ( Fig. 1(b)) is exploited in image processing (see e.g., [25], [26]).

B. MICHELSON CONTRAST
Among the optical contrasts, the Michelson contrast [23] is one of the most well-known contrast measures. This contrast was first introduced in order to quantify the visibility of the interference fringes such as in sinusoid gratings. Indeed, Michelson contrast can also be expressed as a function of reflectance of a thin layer producing the interference fringes. The Michelson contrast is computed by where L min and L max correspond to the minimum and maximum luminance values in the image, respectively. Following a similar concept to the Michelson contrast, other variants of Michelson contrast were proposed for simple images by replacing L max and/or L min with L avg which corresponds to the average luminance value in the image. As an example, King-Smith and Kulikowski [27] introduced as their definition of contrast. While Burkhardt et al. [28] measured contrast by Similarly, Whittle's contrast [29] is measured by It is clear that in the case of natural images the definition of the Michelson contrast and its variants would not provide an accurate estimation of the image contrast. In fact, an isolated point, or salt-and-pepper noise, in a given image can lead to an overestimation of the contrast and therefore not reflect the visual aspect of the image.

C. MOON-SPENCER CONTRAST
While for a long time the Weber-Fechner contrast remained the sole reference in this field of work, by taking advantage of the works of Holladay [30] and Hecht [21], Moon and Spencer extended the notion of luminance contrast to the case of an object embedded in a non-uniform luminance background [24], [31]. The main idea behind Moon and Spencer's experiment is to apply Holladay's principle [30] that any non-uniform background may be replaced by another uniform luminance that produces the same perceptual effect (Fig. 2). This leads to the definition of luminance adaptation that can be calculated according to a simplified model (Fig. 3). Based on the work of Moon and Spencer, the luminance adaptation Illustration of the Holladay principle [30]. Both targets produce the same luminance of retinal adaptation.
In Eq. (7), C W corresponds to the just noticeable Weber-Fechner contrast which usually is equal to 0.02, and A is a constant derived experimentally from psycho-visual tests and Hecht's law [31] which is equal to 0.808. From Eq. (7) we observe that the contrast depends on the luminance of the immediate neighborhood and the luminance adaptation is dependent on several geometric and photometric parameters. Compared to the Weber-Fechner experiment, this definition seems more suited for natural images and has been used for image smoothing and other applications [32]- [35]. It should be emphasized that it is necessary to adapt this contrast measure, established in the case of continuous optical images to the case of digital images. One very simple geometric consideration allows establishing a rule of choosing the possible window sizes and to calculate the different photometric quantities in the case of digital images [32].

III. DIGITAL CONTRAST MEASURES
In this section, we introduce and discuss different definitions of contrast that could be used in digital image processing and analysis applications. The discussion starts with very basic definitions and proceeds towards the recent more sophisticated contrast definitions based on decomposition models representing the early stages of the HVS, namely the retino-cortical mechanisms. The inherent spatial representation of visual signals, and especially images, captured by the HVS appears as spatial distribution of coherent stimuli and visual tokens grouped as objects, uniform background, and textures on a 2D projected information emanating from the 3D real world. This has prompted researchers to define contrast in the spatial domain. From the spatial information, one can define various measures related to the relative visual signal [32]. Moreover, in images, useful information tends to be localized in space, orientation, and scale of resolution [37]. Therefore, to define a contrast measure, one must take into account the two essential aspects of the mechanisms of vision, the frequency selectivity and the directional selectivity [38].
The existing contrast definitions could be broadly classified into two main categories i.e. global (Section III-A) and local (Section III-B) contrast measures. These contrast measures could be computed in the spatial domain, frequency domain, and even multi-resolution or multi-scale representations. To the best of our knowledge, there is no well-established model or well-defined experimental findings on spatio-temporal contrast and so we will not be discussing such aspects. In the following sections, we start discussing the different contrast measures in digital images and highlight their strengths and shortcomings.

A. GLOBAL CONTRAST MEASURES
Global contrast measures are based on global characteristics of an image (i.e., maximum and minimum luminance values). A global contrast measure is usually defined as a global ratio between the darkest and the brightest pixels of an image. The global contrast measures depend on the absolute luminance values rather than considering the luminance of surroundings and do not take into account the local spatial features of the surround. Since images are non-stationary signals, the contrast is very often defined locally to account for the non-stationary nature of the image signals. In the following sections, we discuss global contrast measures defined in the literature.

1) OPTICAL CONTRASTS
The Weber-Fechner and Michelson contrasts are the first global contrast definitions widely used in many applications such as CE, IQA, and quantization. As mentioned in the previous section, these contrast measures are defined in a limited and specific context for the study of HVS mechanisms (for JND analysis in W-F model) or physical phenomena (interference in the case of Michelson experiment). Although these contrast measures are simple and limited to specific experiments they have been exploited and adapted in many image analysis and processing methods [22], [25], [26], [28], [39], [40]. However, these contrasts do not integrate the frequency and directional selectivity of the HVS and do not take into account the color and multi-scale aspects that play a prominent role in visual perception.

2) LILLESAETER CONTRAST
Lillesaeter contrast is considered as the first attempt to take into account both photometric and geometric aspects in the computation of a contrast measure. Two definitions of the contrast have been introduced in [41]. In the first definition only the luminance was taken into account while in the latter it integrates the form of the perceived object. Keeping in mind the asymmetry of the Weber-Fechner contrast [3], [20], it is clear that the negative and positive Weber-Fechner contrasts with the same absolute value are not perceived in the same manner by the HVS. The Lillesaeter contrast is then measured by where L O and L B correspond to the average luminance of the object and the background, respectively. In the case that L O and L B are very close to each other (at approximately 0.02) the Lillesaeter contrast is equivalent to that of the Weber-Fechner contrast measure As mentioned earlier, the second definition of contrast where the geometry of the contours of the perceived object is integrated was also proposed in [41]. In practice this measure is not easy to use since it requires prior knowledge of the object contours and computation of the curvilinear integral along the boundaries of the objects contained in the observed image.

3) CALABRIA AND FAIRCHILD CONTRAST
Calabria and Fairchild introduced two global measures for perceived contrast in color images in the CIELAB 1976 color space [42]. The first is called Reproduction Versus Preferred (RVP) contrast and the second is the Single Image Perceived (SIP) contrast. These two concepts are based on the observer perceptual contrast preference. The contrast model parameters are determined through psycho-physical experimental investigation on the effects and the influence of three key factors, namely the relative lightness, chroma, and sharpness on the observer preference of the perceived image contrast. Two contrast metrics are then defined. The RVP contrast is determined using a simple regression to fit the empirical model to the experiment data and image pair characteristics where In Eq. (10) κ C , κ L , and κ S are experimentally determined parameters related to image chroma, lightness, and sharpness, respectively. The second contrast metric, SIP is defined in a similar way but by considering single image characteristics and observer preference data In Eq. (11) κ C , κ L , and κ S are the standard deviations of image chroma, lightness, and high-passed lightness respectively. Note that the key parameters of the models are determined experimentally using simple linear regression models which are not justified. The other limitations of the experiments and the model is the absence of any parameter related to the multi-scale aspect, the lack of frequency, and directional selectivity.

4) STATISTICAL AND STRUCTURAL INFORMATION BASED CONTRAST
Some descriptors based on statistical and local structural information has been proposed for various applications ranging from texture and image classification to face detection. While these measures could not be fully considered as a contrast measures they express or contain some local information related to the contrast. The following measures could be considered as pseudo-contrast measure but they do not satisfy the main criteria of a contrast measure. Indeed, these measures being invariant to luminance change could not be considered as contrast as defined in psycho-physics and neurosciences fields.

a: HARALICK CONTRAST MEASURE
In [43], Haralick et al. proposed a set of texture descriptors based on the gray level co-occurrence matrix (GLCM) computed from the luminance component of a digital image. Among these texture descriptors, they computed a global contrast as: (12) where i and j are the gray levels of adjacent pixels in a defined neighbourhood; and p ij is the joint probability mass function derived from the GLCM. This measure is supposed to capture the average local variations and spatial dependence of the pixels, i.e. gray-level transition probabilities. In this contrast measure, only short inter-pixel correlations are taken into account. Furthermore, this measure does not explicitly take into account the spatial frequency content in the image. Moreover, it does not capture the relative variations that is related to the visibility of the details.

b: ROOT MEAN SQUARE (RMS) CONTRAST
A relatively simple measure of the global contrast in natural images, called RMS contrast has been introduced by Bex and Makous [44]. The RMS contrast is based on the standard deviation of the image luminance. This measure has been proven to be a reliable metric in predicting the threshold of human contrast detection in natural scenes. A slight modification of RMS contrast has been introduced in another study by Frasor and Geisler [45]. The global contrast is given by: where N is the total number of pixels in the patch, L i corresponds to the luminance of the i th pixel, and w i represents the weight of the raised cosine windowing function at the i th pixel.
The patch luminance is given by: In the study, four different sizes for the patch radius were used (8,16,32, and 64 pixels). This simple definition of global contrast is practical and could be attractive to use in real-time applications. Nevertheless, it does not contain any information on the spatial frequency content of the signal and could not be a good candidate for some applications such as IQA or perceptual coding.

c: CONTRASTS OF RANDOM STIMULI
In an interesting experimental and theoretical study, Moulden et al. [46] introduced six contrast measures to analyze gray-scale random dot images. These categories of random gray-scale images were synthesized by generating random gray levels of the stimuli using three gray level distributions (uniform, binomial and inverse binomial). These global contrast metrics are described below • SD contrast: Standard deviation of the stimulus luminance • SDLG contrast: logarithm version of SD contrast • SAM contrast: the space average Michelson contrast • SALGM contrast: the logarithm version of SAM • SAW contrast: the space average Whittle contrast • SALGW contrast: the logarithm version of SALGW contrast In Eqs. (16)-(21) L i and L j are the discrete luminance values for pixels i and j, p i and p j are the probability of occurrence of those luminance values, and L represents the space average luminance of the stimulus. It should be noted that these metrics have some advantages over other conventional measures such as the Michelson or Weber contrasts in that they incorporate the contributions of all stimuli. The introduction of log in some measures allows to implicitly incorporate the non-linearity of the HVS response to visual stimuli.

d: MUTUAL INFORMATION BASED CONTRAST MEASURE
Building on the texture features introduced by Haralick for the analysis of structural information in greyscale images [43], a contrast measure based on mutual information has been introduced in [47] to quantify the side effects that may result from the contrast enhancement process. This contrast measure is derived from the joint probability mass function of a gray level co-occurrence matrix and expressed as where p xy (i, j) is the joint probability mass function of the luminance channel, whereas p x and p y represent the marginal probabilities computed from the GLCM. Compared to the 1 st order statistics based measures such as entropy, this approach is a better option since it takes into account the spatial correlation among the pixels using the GLCM. While this contrast measure is simple to compute, however, it does not provide information directly related to visual perception as it is purely based on statistical analysis of the signal values distribution.

5) MATKOVIC et al. CONTRAST
Matkovic et al. contrast is a global measure of the overall image contrast as perceived by the HVS [48]. This contrast measure is very similar to the RAMMG contrast proposed by Rizzi et al. [49]. Based on a multi-resolution decomposition scheme and a weighting process, this contrast measure takes into account the multi-scale characteristics of the HVS. Based on some psycho-physical experiments, the weighted average of the local contrast computed at different resolution levels is then represented as the Global Contrast Factor (GCF) [48] In Eq. (23), w k and C k represent weights and average local contrast of the image at the given resolution k respectively, and N represents the number of resolution levels in the calculation. The local contrast for the pixel (i, j) at resolution k is computed as the average of the absolute differences values of that pixel with its four nearest neighbors In Eq. (24), W k and H k correspond to the image width and height at the k th resolution, respectively. Note that by taking into account the multi-resolution analysis which is more plausible, this contrast measure seems to be more informative than other global contrast measures as it takes into account both the local variations of the signal and the multi-scale aspect.

6) POTENTIAL CONTRAST MEASURE
The Potential Contrast (PC) measure for digital images which is based on the Weber-Fechner's model where the image is considered as consisting of a background and a foreground and where the main objective is to segment the image to two classes of pixels was introduced in [50]. The PC measure proposed by Shaus et al. [50] is based on both the Weber-Fechner VOLUME 8, 2020 image model and the CMI index measured by where µ b and µ F are the average luminance of the background and foreground, respectively. The original idea behind this measure is to construct a contrast measure that verifies three essential properties. First, the background/foreground separability which the measure should be able to quantify in an unambiguous way (the photometric difference between the background and the foreground). Second, the non-sensitivity to gray-scale transformations such as histogram equalization and brightening. Third, the invariance with respect to some invertible gray-scale transformations. The PC measure associated to a given image I is constructed through an optimal transform (T opt I ) satisfying the three stated properties. The PC contrast associated to the image is calculated by where o is the transformation composition operator. Note that the construction procedure for PC can be generalized to indexes other than the CMI index. Although this measure appears to meet clearly established mathematical concepts and criteria set out by the authors, it performs well in the case of two-class image segmentation applications and it cannot be considered as a measure of visual contrast as such. Indeed, the PC measure is constructed based on mathematical criteria around the optimization of a gray-scale transformation. This measure also does not contain the frequency and directional aspects of the structures in images.
To conclude this section on global contrast measures, it is worth noticing that one of the advantages of global contrast measures is that only one single value is associated to each image. Another advantage of this category of contrast measures is its ease in computation and simplicity in general. One of the critical limitations of these measures is that they are inappropriate to natural images containing different structures with various spatial frequencies and directions which cannot be taken into account in a single global measure. One solution to adapt these global contrast measures to complex and natural scenes is to compute the contrast locally at each pixel by using a sliding window in the whole image.

B. LOCAL CONTRAST MEASURES
Contrast measures that take into account the neighboring pixels in the computation of contrast are referred to as local contrasts. To overcome the shortcomings of global contrast measures, many local measures have been developed in the literature. In the following, we discuss local contrast measures proposed in the literature starting with the pioneering works done, especially by the vision and optics research communities [51], [52], [56], [71], [72].

1) FOURIER TRANSFORM BASED CONTRAST
To the best of our knowledge, the first formulation of image contrast in the spatial frequency domain was the work of Hess et al. [51]. The underlying idea in the expression of contrast in the transform domain is to take into account the frequency content in the image, especially in the case of complex natural scenes. Hess measured contrast in the Fourier domain by In Eq. (27), u and v are the horizontal and vertical spatial frequency coordinates, A(u, v) is the amplitude of the Fourier transform of the image, and A(0, 0) corresponds to the DC or zero-frequency component. This Fourier Transform based contrast is the first definition of local contrast defined in the frequency domain. However, one of the limitations of this contrast is that a division by the high-energy DC component of the whole image can mask high-frequency components especially in natural and complex images. Such an approach could result in underestimating the visibility of fine details of low energy. Another limitation of this contrast is the lack of multi-scale and directional selectivity.

2) GORDON AND RANGAYAN CONTRAST
To the best of our knowledge, the definition of the local contrast and its use in digital images was introduced for the first time by Gordon and Rangayyan [52]. This contrast is defined by analysing the image using an odd size sliding window (W ij ) centred at pixel (i, j). This contrast corresponds to the relative variation of the gray level in the local analysis window, between the near surround and the far surround. The Gordon and Rangayan contrast is similar to the foveal image model in the Moon and Spencer contrast where f c (i, j) and f s (i, j) are the mean gray levels in the local window and the surround respectively. Note that in the case of the smallest window size of 3 × 3, f c is nothing but gray level at the center pixel (i, j) and f s is the mean gray level of the eight neighbors. As noticed in [73], this measure is noise sensitive and does not integrate the spatial frequency information and other relevant characteristics of the HVS such as the directional selectivity and the multi-scale aspects.

3) BEGHDADI AND LE NEGRATE CONTRAST
Inspired by the work of Gordon and Rangayyan [52], Beghdadi proposed a contrast measure based on local edginess information in the image [53]. An in-depth analysis of this contrast measure, and in particular its sensitivity to noise in CE can be found in [73]. Some variants of this measure have been proposed in the literature for CE of an image [74]- [77]. The measure is based on the fact that compared to other existing structures, the HVS is more sensitive to the edges in the observed image. Therefore, the main idea is to quantify the local sharpness of the contour by measuring the visibility of the salient features through a sliding window. This visibility measure is considered as a local contrast measure and is expressed as a ratio measuring the photometric distance between a given pixel and the contour in the window analysis. For each pixel at (i, j), centered at the odd-sized W ij window, the mean intensity level of the contours, or mean edge gray level, is calculated by In Eq. (29), f (k, l) corresponds to the gray level at point (k, l) for image f and ( kl ) is an increasing function of the gradient operator at (k, l). An example of a simple function is a positive integer power of the gradient magnitude (i.e. n kl , n > 0). The robustness of this estimator has been studied in detail in [78]. It was also demonstrated that the pseudo-gradient obtained by combining two responses of the Sobel operator, to compute kl , provides the most robust estimator for additive Gaussian white noise [78]. Once the level E(i, j) is estimated, the local contrast associated with the pixel (i, j) is expressed as We should note that this contrast measure does not take into account the frequency selectivity nor the directional selectivity aspects. Furthermore, when used in CE it may introduce halo effects around the edges which is the result of over-enhancement [47], [79].

4) JOURLIN et al. CONTRAST
Jourlin and Pinoli [54] introduced a contrast measure based on a physical model incorporating the Weber-Fechner law and the notion of image contrast introduced by Kohler in his gray level thresholding method [80]. A new image representation and processing model called Logarithmic Image Processing (LIP) was then introduced to overcome some limitations on the classical description of images and particularly the Michelson contrast measure. In the LIP contrast which is given by X 1 and X 2 correspond to two pixels in the image at positions (x 1 , y 1 ) and (x 2 , y 2 ) respectively. Note that while this measure involves two pixels and could be considered as a distance, it does not take into account the spatial information around the neighborhood of the pixels. Furthermore, the frequency content and the directional selectivity are not considered in this measure.

5) TOET CONTRAST
Using the Burt and Adelson pyramid decomposition scheme [81], Toet introduced for the first time a multi-resolution contrast measure in his image fusion method [55]. The contrast is expressed as This contrast measure is the ratio between the relative difference of the Gaussian pyramid component at two successive levels and the low-level component. In Eq. (32), g k (i, j) is the gray level of pixel (i, j) in the Gaussian pyramid at level k.
Note that similar to many of the existing contrast definitions in the literature, this definition is consistent with the Weber-Fechner model. Indeed, the numerator contains the luminance increment corresponding to the band-pass component and the denominator term acts as a uniform background component at the current level of the pyramid. One of the limitations of this measure is the lack of directional sensitivity and color aspects.

6) PELI CONTRAST
Based on the frequency selectivity of the HVS [56], Peli defined an isotropic contrast measure, similar to the one introduced by Toet in [55]. In the method, the image is decomposed into several channels using a bank of isotropic cos-log bandpass filters. In this measure, the image component of the k th channel is calculated by where the impulse response of the filter corresponding to the k th channel is represented by h k and g k corresponds to the filtered image of image f . For pixel (i, j) in the k th channel, the contrast is calculated by which corresponds to the ratio of the luminance in the k th channel and the lowpass component corresponding to the energy captured by the channels below k. In Eq. (34), The Peli contrast is well suited for complex images and is often used in the computation of the quality of encoded images [82]. It should be pointed out that the Peli contrast does not take into account the directional selectivity of the HVS and so, an isotropic bandpass filter is used in its computation.

7) LUBIN CONTRAST
Another band-limited contrast based on pyramid decomposition and inspired by the Toet contrast measure and the Peli model has been introduced by Lubin [57] for the computation of an image distortion measure known as the Sarnoff model which can be used as an IQA metric. In this model, the lowpass component is moved down one resolution level in the pyramid. The local band-limited contrast is defined as the measurement of the differential signal strength of two neighboring low-pass filtered components relative to the low-pass VOLUME 8, 2020 filtered component at two levels down. This contrast measure which is given by also falls in the same category of Peli and Toet metrics and suffers from the same limitation, i.e. lack color aspect and directional selectivity.

8) DALY CONTRAST
A similar approach for defining a band-limited contrast based on the cortex transform has been introduced by Daly in the Visible Difference Predictor (VDP) model [58]. The cortex-based contrast is measured by where k and l are the dom and fan filter indices as defined in the cortex transform. These band-pass filters are defined in the spatial domain and mimic the directional and frequency selectivity of the HVS [83], g kl (i, j) is the pixel intensity in the band (k, l) resulting from the filtering operation where h kl is the cortex filter and g kl is the average energy in the band (k, l). However, as pointed out by Daly, the mean of the subband component g kl is zero leading to an indefinite contrast value. Daly proposed to use the base band energy in the denominator to fix this problem resulting in the contrast to be calculated by Note that the baseband g K is nothing but a Gaussian low pass filter. It is worth noticing that Daly has introduced four modifications in the original cortex transform as defined by Watson [83]. Unlike all the previously discussed measures, the definition of the Daly contrast integrates both frequency selectivity and directional selectivity through the cortex transform and has been used in many IQA metrics. The only relevant aspect that is not considered in this measure is the color.

9) JOLION CONTRAST MEASURE
Inspired by the work of Burt and Adelson [81] and Toet [55], Jolion introduced the multi-resolution contrast or pyramidal contrast [59] based on the definition of contrast by Beghdadi and Le Negrate [73]. In the proposed approach, the image is analyzed by a multi-resolution decomposition of a Gaussian pyramid. At a given resolution level (r), the local contrast at pixel (i, j) which is centered at the window W ij is equal to In Eq. (40), g ij corresponds to the gray level at resolution r, obtained by Burt and Adelson pyramid decomposition of the original signal [81], and e (r) ij represents the coefficient from the associated Laplacian pyramid [59]. A contrast map image C r is then derived at each resolution r.
The objective of analyzing the contrast at different resolution scales is to simulate the effect of the distance between the image and the observer. Another similar contrast definition based on wavelet transformation has been proposed in [84]. This measure introduces the notion of contrast at multi-scale and provides the means to analyze the image at different levels of details. These multi-resolution contrast measures seem to be efficient in capturing the relevant features at different scales but lack the directional selectivity.

10) POWER-LAW CONTRAST MEASURE
To overcome the limitation of the Weber-Fechner contrast, Frese et al. used a power-law to define the contrast in the CIE XYZ color-space [60]. This contrast takes into account the physiological aspects of the HVS and is measured by where Y is the luminance of the foreground stimulus and Y B is the uniform background luminance. It is worth noticing that the Weber-Fechner contrast value could diverge when the background luminance is close to zero. Whereas, the measure proposed by Frese et al. in [60] allows to avoid such disadvantages and therefore it is more consistent with perceptual human vision. Furthermore, a multi-resolution contrast is derived from this simple definition by analyzing the image using a pyramid decomposition. The contrast at a given level is obtained by subtracting two consecutive low-pass filtered components where Y k denote the luminance component at pyramid level k (k = 0, K − 1), and K − 1 is the coarsest level of resolution. This contrast definition is extended to the opponent color space in which two opponent color-contrast channels are then measured by where C X and C Y are power-law contrasts in X and Y (interpreted as the difference between ''red'' and ''green'' contrast channels. Similarly, the difference between ''yellow'' and ''blue'' contrast channels is expressed as These contrast definitions have been used in the design of an image similarity measure [60]. However, it should be noted that none of these contrast measures take into account the directional selectivity property of the HVS.

11) AHUMADA AND BEARD CONTRAST
In the simple vision model for the computation of the IQA proposed by Ahumada and Beard [61], the contrast is defined in two steps. First, the original signal is convolved with a Gaussian low pass filter resulting in a blurred image A second Gaussian low pass filter is then applied to the filtered image g 1 resulting in where h 1 and h 2 are two distinct Gaussian lowpass filter masks. Finally, the local contrast at each pixel (i, j) is calculated by Note that this contrast is inspired both by the Weber-Fechner definition and the Peli contrast. In other words, this contrast is expressed as the ratio between the band-limited component (g 1 −g 2 ), representing the increment (target), and the low pass component (g 2 ), that plays the role of the background. Note that this contrast does not take into account the frequency and directional selectivity of the HVS.

12) WINKLER AND VANDERGHEYNST CONTRAST
Winkler and Vandergheynst [62] have pointed out the limitations of the Peli's contrast [56] and proposed a measure using a directional wavelet decomposition. This decomposition requires the use of 2D analytical filters which is complex to perform in practice (difficulty to extend the Hilbert transform into 2D). This isotropic contrast is based on the computation of summing the squared anisotropic responses. To circumvent this difficulty, non-separable analytical directional filters are used. An isotropic contrast is then measured by where g kl (i, j) is the gray level at pixel (i, j) in the band-limited directional filtered image and g k (i, j) is the filtered signal with the scaling function at scale k. In contrast to the Peli's contrast, this new contrast measure has the advantage of giving a flat response instead of an oscillating response to sinusoid gratings as demonstrated in [62]. This contrast has been proven efficient in highlighting salient features as compared to Peli's contrast. But it lacks the directional selectivity in the final output as it is based on a average process of the anisotropic responses of the subband filters.

13) BELKACEM AND BEGHDADI CONTRAST
A local contrast measure, inspired by the works of Moon and Spencer [24], [31] and enriched by some relatively recent knowledge on the functioning of the HVS, has been proposed for effective low-level image processing [32], [63]. The contrast measurement combines Moon-Spencer's foveal image model and multi-channel analysis. The original image is first analyzed using a directional Gabor filter bank to extract the visual information at different spatial and directional frequency bands. The choice of Gabor filters is motivated by the fact that they perfectly simulate the receptive fields in the cortex and present several interesting properties [37], [85]. The components obtained are then combined linearly to provide the signal where N G is the number of Gabor channels. The linear combination of the different band-limited frequency components are justified by different models of the HVS [86], [87]. Another possibility is to insert a competition stage between the different responses. The local contrast is then given by In Eq. (50) corresponds to the average luminance of the surround, or the immediate neighborhood. In Eq. (51), and C ij corresponds to the set of points of the surround (Fig. 3).
While this contrast seems to include many aspects related to the HVS, it still does not integrate color aspect. Furthermore, by averaging the bandpass responses, the role of the directional selectivity might have been weakened. In the measure, a competitive process is proposed as an option but not clearly explained.

14) TADMOR AND TOLHURST CONTRAST
Tadmor and Tolhurst proposed a local contrast measure based on the Difference Of Gaussian (DOG) model [64] that has been successfully used to describe the responses of retinal ganglion cells (RGC). In this model, the spatial responses of RGC is modeled as the difference of two circularly symmetric Gaussian filters. Two zones are then defined on the receptive field of RGC, namely ''center zone'' and ''surround zone''. The Gaussian kernel associated with the center zone is given by: In Eq. (53), r c is the distance beyond which the sensitivity decreases below 1 e with respect to the peak level. The surround component is represented by another Gaussian filter, with a larger radius, r s , where the central point of the receptive-field is placed in (i, j) and the output of the central component is expressed by and the responses in the surround zone is given by: where f (i, i) corresponds to the input intensity pixel at (i, j), c is the central zone of size [−3r c , +3r c ]x[−3r c , +3r c ] while the surround zone s is of size [−3r s , +3r s ]x [−3r s , +3r s ]. Tadmor and Tolhurst propose the following three options for the computation of the contrast at a pixel (i, j) and The authors also provide a global contrast measure by averaging the local contrast values across the whole image Note that in the original implementation, the DOG contrast measure is calculated by taking the average of contrast (C k (i, j) for k = 1, 2, 3) across 1000 randomly selected pixels in an image for fast implementation. We should point out that none of these measures take into account the directional selectivity. The color aspect is also not considered in these measures.

15) DIRECTIONAL BANDLIMITED CONTRAST
Inspired by Peli's model, Dauphin et al. [65] proposed a new bandlimited contrast measure [65] that takes into account the multichannel behaviour of the HVS. Through different neurophysiological and psycho-visual experiments, the proposed approach takes into account the frequency and directional selectivity of the cortex [83], [85]. The image is analysed using a multichannel Gabor decomposition. A nonlinear combination of the directional responses is then used to calculate contrast at each pixel. In the experiments, local directional bandlimited contrast has been analysed on natural and synthetic images. In the first step of this approach, the image is decomposed into frequency and directional subbands. For each band centered at a given frequency, ω k = 1.3, 2.7, 5.4, 11, 22 and 44 cycles/image and the local directional bandlimited contrast is expressed as follows In Eq. (61), g kl (i, j) is the gray level associated to the frequency sub-band ω k and to one of the four directions (0, π/4, π/2, 3π/4) represented by l. In the equation, the normalization term g k (i, j) represents the total energy of the background below the k th sub-band which is obtained by filtering the original image by a Gaussian filter with a standard deviation, σ k = 0.75ω k . Unlike Winkler and Vandergheynst method [62], here the directional structures are highlighted and contribute in the final contrast measure. However, although this contrast measure integrates both directional and frequency selectivity, it does not consider the color aspect. Furthermore, there is no findings on the modeling of the HVS to justify the use of the max operator in the computation of the final contrast measure. In fact, although the use of the max operator in this definition seems to be inspired by some models of the nonlinearity of the HVS and in particular the model proposed by Malik and Perona in [86], to our knowledge there is no well-established psychovisual data that could support such a reasoning. However, it was found that the spatial orientation and the color have an impact on the CSF [88], [89].

16) NEURAL MODEL BASED CONTRAST
The notion of contrast in neural computation models is expressed as the response of the neuron to visual stimulus. One definition that could be adapted and used in digital image processing and analysis is derived from the famous Grossberg equation for modeling the rate of change in neuronal activities [90], [91]. The contrast expression which can be derived from the shunting equation solved at the equilibrium state [66] is then expressed as a function of two inputs, an excitatory part and an inhibitory part. One simplified expression of this kind of contrast for digital images is given in [66] and could be expressed as where I σ c (x, y) is the center input stimulus signal (the center has one pixel width for practical use in digital image processing) and I σ s (x, y) is the surrounding input stimulus given by and where h σ c and h σ c are two Gaussian kernels represented by 7 × 7 and 19 × 19 convolution masks respectively and α, β, and γ are constants to be tuned experimentally. Note that this contrast is less known and its use in image processing applications is very limited as it does not integrate the frequency and directional selectivity of the HVS.

17) AGAIAN AND PANETTA CONTRAST MEASURES
Panetta and Agaian developed various simple Contrast Enhancement Evaluation (CEE) measures based on the computation of a global index derived from some local measures related to contrast and image gradient [39], [92]- [96]. These CEE measures are mainly inspired by the Michelson and the Weber-Fechner contrast measures. According to recent critical studies conducted on the performance analysis of CEE metrics [97], [98], the most consistent metric related to the HVS namely Absolute Measure of Enhancement by Entropy (AMEE) is being discussed here.
The AMEE is a no reference metric proposed in [39] based on a contrast measure using the dynamic range (min-max values) pixel value within a block. The image is first divided into non-overlapping blocks of the same size (8 × 8). Then, AMEE is computed based on the minimum and maximum pixel values in each block, respectively. Since log of ratios of maximum and minimum intensities within each block can be written as a difference, AMEE may represent the dynamic range of the image. The local contrast at pixel (i, j) is computed by where I max (i, j) and I min (i, j) are the maximum and minimum pixel intensities within the window W ij centered at pixel (i, j) respectively, is the constant to control the division by zero, and α is an exponent that controls the enhancement effect. This local contrast is used in the computation of the global measure AMEE for CEE. Note that none of the measures proposed [39], [92]- [96] take into account the frequency and directional selectivity of the HVS.

18) DIFFUSION-BASED CONTRAST
An interesting and less known definition of the local contrast in the context of CE using anisotropic diffusion model [99], has been introduced in [100]. The equation governing this spatial temporal phenomenon of diffusion at different scales is nothing other than the model introduced by Malik and Perona for image enhancement [101]. During the iterative process of diffusion, the local contrast at a scale t representing time is calculated by where L(x, y, t) is the luminance of the pixel (x, y) and L(x, y, t) is the luminance of the local surrounding background at scale t, respectively. Defining a contrast evolving over time is interesting and represents a first step for moving towards the spatio-temporal analysis of visual information. However, this contrast does not incorporate the true concept of time. It is rather the number of iterations in the numerical diffusion model and abusively identified as the common time parameter. Furthermore, this contrast does not integrate the spatial frequency nor the directional analysis of the signal.

19) MULTISCALE COLOR CONTRASTS
Interesting contrast measures have been proposed for color images that take into account characteristics of the retino cortical mechanisms of the HVS [68], [69], [102]. These measures are introduced in the following section.

a: RAMMG CONTRAST MEASURE
Rizzi et al. [49] proposed a multiresolution contrast measure for color images based on pyramid like decomposition. Here, the image is first decomposed into various levels of resolution in the CIELAB color space. A local contrast is then computed at each pixel by taking the average value of the absolute differences between the luminance of that pixel and its eight neighbors. The local contrast at pixel (i, j) at the k th level of resolution is expressed as.
where ij is the 8-neighborhood of the pixel (i, j) and mn is the weight associated to the pixel (m, n) and given by: A global contrast at each level of resolution is then computed by averaging the local contrasts as follows.
where W k and H k are the width and height of the image at the k th resolution level. The image resolution is halved in each subsequent level, where W k = W k−1 /2 and H k = H k−1 /2 represent the widths and heights of the image at the k th level respectively.
The global contrast C R of the image is finally computed by taking into account all the global contrasts at different levels of resolution.
where K is the total number of resolution levels.

b: RETINAL-LIKE SUBSAMPLING CONTRAST (RSC)
Simone et al. proposed a multi-level RSC measure [68] which combined the multilevel approach for the RAMMG [49] in Section III-B19 and Tadamore DOG based contrast [64]. The summary of the steps are as follows: Step 1:Subsampling the original image and transformation into the CIELab color space.
Step 2:For each pixel (i,j) and each level, determining the DOG neighborhood local contrast for luminance and chrominance channels separately. VOLUME 8, 2020 Step 3:Recombination of the local contrast maps to have the local contrast measure for the luminance and chrominance channels separately.
Step 4:The weighted combinations of the three local contrast measures from luminance and chrominance channels is used to calculate the RSC measure.

c: WLF CONTRAST MEASURE
Simone et al. [69] used an antialiasing filter in the subsampling and a weighted recombination of the local contrast maps. Considering the facts that each level has a different contribution, the contrast is calculated by (72) where K is the number of levels, C ik is the mean contrast at level k, and i indicates the color channel. The new parameter, λ k is the weight assigned to each level k. The overall final measure is the given by where α, β, and γ are the weights of each color channel. This measure is not only limited to the CIELAB color space but can also be extended to other color spaces such as XYZ and RGB. Unlike contrasts measures discussed previously, this series of measures integrate both the multi-resolution and color aspects. Nevertheless, none of these contrasts involve the directional selectivity of the HVS. Another aspect that is poorly supported in these measures is the way in which different features are combined using a weighted averaging operation.

20) DIRECTIVE CONTRAST IN NSCT DOMAIN
An interesting contrast measure defined in the Non Sub-sampled Contourlet Transform (NSCT) has been introduced in [70] for multi-modal image fusion. The measure is inspired by the directive contrast introduced in [103] in which four-directive contrast measures are defined in the wavelet domain corresponding to the approximate, horizontal, vertical, and diagonal subbands, respectively. The new definition of the directive contrast, as defined in [70] is given by where g sθ corresponds to the low frequency subband at scale s and direction θ in the NSCT domain. The exponent α is a constant which is experimentally determined and has a value in the range of [0.6, 0.7]. The term sθ (i, j) is a fusion measure called Sum-Modified-Laplacian (SML) and expressed as where m × n is the size of the analysis window. The term ∇ 2 g sθ (k, l) contains the local variations of the signal at scale s and direction θ expressed as the response of the modified Laplacian introduced by Nayar and Nakagawa in [104] and calculated by We should point out that this contrast is based on a multi-scale and multi-direction transform and is limited to only three directions (horizontal, vertical, and diagonal).

C. DISCUSSION
A brief summary of the contrast measures covered in this section is provided in Table 2. It reveals that none of the discussed contrast measures integrate all the characteristics of the HVS that are relevant for analysing natural images through the perceptual contrast. It could be also noticed that most of the contrast measures are computed in the spatial domain except Hess's contrast metric. However, some of the contrast measures could be expressed in the transform domain, it is the case for example for the contrasts by Peli, Daly and Lubin. Note that the only contrast which incorporates the viewing distance in an explicit fashion is Moon and Spencer model. Some other contrast measures implicitly integrate this parameter through the multi-scale or multi-resolution representation of the contrast measure.
Other local descriptors such as Local Binary Pattern (LBP), used for texture analysis and face recognition [105], [106], are somehow considered as a measure of local contrast as they allow to discriminate the different patterns and details at different scale and direction. However, this kind of local texture descriptors could not be fully considered as a measure of contrast as it does not capture the relative variation of the image signal as expected from any contrast measure. Moreover, LBP descriptors are invariant to luminance change and therefore could not capture the relative variation of luminance as required by contrast measure. Indeed, the LBP descriptors contain spatial structure information but do not address the contrast as noticed by the authors themselves who proposed to combine the LBP with a local contrast in order to define the LBP/C descriptor [106]. In another study authors combined LPB with a local contrast measure for face recognition. The authors also implicitly recognized that LBP cannot be considered as a contrast measure [107].

IV. SUMMARY AND LESSONS LEARNED
Through this chronological study on the measurement of visual contrast, and the underlying models and experimental studies, it is clear that taking into account all the relevant factors and parameters that come into play in the mechanisms of visual perception related to the notion of contrast in one single expression is a challenging task. Nevertheless, most contrast measures use a simple version of the typical framework shown Fig. 4 that tries to take into account the most relevant characteristics of the HVS. While contrast measures do not necessarily use all the elements shown in the framework most if not all measures can be linked to it. We should point out that despite all the hard work done on introducing different contrast measures, the question raised by Haun and Peli still exist, ''how does a human observer evaluate the contrast of a complex image?'' [114].
From the studies we can see that the notion of contrast highly depends on the point of view from which one apprehends the image and for what application it is designed for. If we strictly limit ourselves to only visual aspect, it is clear that a measure that integrates the maximum number of parameters and characteristics related to the mechanisms of visual perception is the most appropriate approach. A contrast measurement could integrate the optical effect of the pupil, the irregular spatial sampling at the level of the retina, the transduction of light into a neural signal by the photo-receptors (color aspect), the decomposition of the signal by the families of cells of the cortex according to VOLUME 8, 2020 frequency and directional bands, and finally a last stage which is poorly documented and which concerns fusion and decision.
Our ability to perceive visual patterns is linked to CSF which is one of the most important characteristics of the HVS. From the first studies conducted by Weber-Fechner on the JND concept which is associated to low-level features, i.e. contrast in the HVS, different experimental and theoretical studies on modeling the CSF has been performed. Contrast can also be closely linked to visual masking which is another major phenomena in the HVS. Different studies which have used CSF for analyzing and modeling visual masking have focused on how it can influence perceptual contrast [22], [48], [117]. Quantifying masking which refers to influencing the visibility of a region in the image (visual target) in the presence of another visual stimulus which is referred to as masker normally requires the computation of a contrast measure. In general, masking models only use simple contrast measures such as the Michelson, Weber-Fechner, and RMS contrast [118].
Through this critical analysis on the existing contrast measures, it becomes important to answer some relevant questions. One important question is: what are the most relevant visual signal characteristics that the contrast measure should capture? We should point out that to the best of our knowledge none of the published work addressed this issue properly. However, some interesting studies have been dedicated to address this critical question [66], [113]. Peli conducted a thorough experimental study in 1997 on how to define the contrast measure [113]. To be consistent with the findings on the human visual perception the study provides guidelines confirming that computational contrast metrics should take into account multiscale aspects [4].
To illustrate the most relevant features of the visual contrast some representative contrast measures for natural images, namely the Peli contrast (Fig. 5(b)-(e)), directional bandlimited contrast (Fig. 5(f)-(i)), edge based contrast (Fig. 5(j)-(m)), and wavelet contrast (Fig. 5(n)-(q)) are shown. Through these images, it is apparent that the notion of contrast is highly related to the visibility level of the salient features in the image. These images also illustrate the importance of the edginess information as highlighted through the edge-based contrast map (Fig. 5(j)-(m)), the frequency selectivity (Peli contrast maps) (Fig. 5(b)-(e)), the directional selectivity (Fig. 5(f)-(i)) and finally the multi-scale aspect as highlighted through the wavelet-based contrast map (Fig. 5(n)-(q)). For example, it could be noticed that directional structures are more highlighted in Fig. 5(f)-(i), and the edges are more highlighted in the edge based contrast map (Fig. 5(j)-(m)). Unlike saliency maps (where ground truth data is available), in case of contrast, there is no clear and objective criteria to use as ground truth data to compare the proposed contrast maps with. Although, some attempts have been made in the study of different measures of contrast in digital images [68], [69].

V. SELECTED APPLICATIONS OF CONTRAST MEASURES
In the previous section, we provided a complete overview on contrast measures proposed in the literature to date. From the overview it seems that there is no universal definition for Note that in the case of the Edge based and Wavelet based contrast maps bright regions correspond to the low contrast and black for the high contrast ones. In the case of the Wavelet based contrast maps we have also up-scaled the image for better presentation. VOLUME 8, 2020 contrast and so for a given application, one has to choose one that is most suitable. In the following, we discuss selected applications of contrast for computing and representation of various visual information. We also provide some guidelines for selecting the most appropriate contrast definition for each application or suggest possible directions. While contrast has been introduced in multitude of theoretical and applied research, here, we limit our study only to image quality assessment and enhancement, image segmentation, watermarking, and image coding and quantization. We also discuss few other applications where contrast based measures and other related image signal descriptors are used.

A. IMAGE QUALITY ASSESSMENT (IQA)
IQA is an active field of research [119]- [126]. Measuring the objective quality of digital images is a complex process involving a series of low-level and high-level processes in the brain. A plethora of IQA methods have been developed during the last three decades ranging from classical natural images to pictures of ancient manuscript documents and paintings [127]- [130]. One of the most crucial factors used in the design of IQA metrics is visual contrast. The contrast measure introduced by Peli [56] (Section III-B6) is widely used in the computation of objective image quality measures [8], [82], [131], [132]. Ajagamelle et al. [133] used the DOG-based contrast from Tadmor and Tolhurst [64] (Section III-B14) in a multi-level approach inspired by Simone et al. [134] for image quality assessment. The DOG-based contrast has also been applied to measure quality in terms of contrast in Magnetic Resonance Imaging (MRI) [135]. Another very interesting and less investigated area of research, where the notion of contrast plays a prominent role is the objective evaluation of CE methods. One example is in medical CE [136], [137]. Sdiri et al. [137] used among others the Absolute Measure of Enhancement (AME), which is based on the Weber-Fechner contrast (Eq. (1)) and the Michelson contrast (Eq. (2)), to evaluate their enhancement technique. Given the multitude of CE methods developed, it becomes practically difficult to objectively select the best method for a given application. In recent studies, some objective metrics based on the notion of contrast have been evaluated on dedicated databases for CEE [97], [138], [139], which is shown to be a challenging task.
It is important to point out that the measure of contrast plays a prominent role in the design of HVS-inspired IQA metrics. Measure of contrast has been commonly used to define the contrast visibility threshold, both for luminance and chrominance. Contrast measures are also important when calculating visual masking thresholds [117], [118], which have been used to predict quality [140]. Contrast measures have also been employed for estimation of the quality score, and inspired strategies for quality pooling [141].

B. IMAGE QUALITY ENHANCEMENT
In this section, we discuss the application of contrast measures in the context of perceptual denoising and CE. Here, we focus on a number of representative methods of these two applications and discuss few recent methods.

1) DENOISING
Any denoising operation naturally results in blurring and loss of fine details in the image and so it is essential to reduce such artifacts when introducing any denoising or smoothing filter. Despite the considerable number of published image denoising methods [142] and the fact that the quality of noise and artifact reduction is very much related to their visibility and therefore to contrast, it is surprising that there are relatively few denoising methods based on contrast measure. For this reason we will limit ourselves to discussing few filtering methods based on contrast measure. A gray level image filtering approach based on the concept of contrast entropy has been proposed in [143]. The contrast defined in this method is based on the adaptation of the Weber-Fechner contrast measure to digital images. Later, another denoising method based on the Moon and Spencer contrast model and Gabor filter was proposed in [32]. In this method, based on the contrast measure defined in Eq. (50), the noise is considerably reduced and the contours and other fine details are not significantly affected. The advantage of using contrast as a selection criterion and to use adaptive filtering makes it possible to avoid degradations and to achieve a treatment where the visual aspect is exploited in the design of the method. An interesting method for color image denoising based on some notion of the HVS and the wavelet transform has been proposed in [144]. The study takes advantage of the CSF and the masking effect but it is not explicitly based on any of the contrast measures. Recently, a denoising method based on local contrast and inspired by the idea developed in [143] has been proposed in [145].
From this brief overview on the use of contrast measure for denoising it appears that there is no clear model linking the contrast measure to the noise distribution. However, the visibility of the noise is related to the contrast measure and the two spatial measures based on Weber-Fechner or Moon-Spencer model are more suitable for denoising. In the case of the wavelet-based or multi-resolution denoising approaches, the most promising measures are those proposed by Winkler and Vandergheynst [62], i.e. the isotropic contrast Eq. (48), and by Bhatnagar et al. in [70] expressed in Eq. (74).

2) CONTRAST ENHANCEMENT
CE is considered to be one of the primary concerns in the field of low-level image processing. To the best of our knowledge, the pioneer work on CE is the work of Kovásznay and Joseph [146]. CE methods can be roughly classified into two categories, direct methods which are designed and based on the contrast measure and indirect measures which are not directly based on contrast measures [73]. From the literature on CE, it appears that the direct method of Beghdadi and Le Negrate [73] and its variants are widely used for CE [73]- [77]. This measure has been also extended for CE of stereo images by incorporating the depth information [147].
The common idea behind such direct methods is to amplify the local contrast C ij into C ij using a nonlinear transform and then derive the corresponding pixel intensity value. Whereas, indirect methods act on the image signal through some local or global quantities such as gray level histogram [148] or local entropy [149]. While the Beghdadi method and its variants do not take into account the spatial frequency content of the image signal when computing the contrast, Tang et al. [150] proposed a method based on the Peli contrast measure in the DCT domain. Another similar approach based on a multiscale contrast measure defined in the wavelet domain has been proposed in [151] for mammography image CE.
In the case of direct methods, the local contrast measure provided in Eq. (30) and its variants are widely used and seem to be efficient. The only limitation is due to the fact that it may produce over-shooting around the edges which is linked to the aggressive enhancement as expressed in the measure [79]. The Peli measure as defined in Eq. (34) has been proven efficient for enhancing the contrast in the transform domain [150]. The wavelet-based contrast measures defined in Eq. (48) and Eq. (74) are also good candidates for CE in multi-scale approaches. Besides these three types of contrasts, based on our experience, there is no other relevant contrast measure that could be more efficient for CE purposes.

C. IMAGE FUSION
Image fusion has many applications ranging from medical imaging, remote sensing imagery or multi-sensor based video surveillance, to name a few [152], [153]. One of the most promising approaches for image fusion is to exploit the perceptual contrast [154], [155]. Many image fusion methods based on contrast has been proposed in the literature [156], [157]. Multi-scale contrast measures seem to be the best option for image fusion [158]. This is due to the fact that subtle details and salient information are better captured by contrast maps. The contrast measure is then used as a weighting function in the fusion process. This strategy has been used in an efficient multimodal medical image fusion scheme based on the directive contrast measure defined in Eq. (74). This contrast measure has been also extended to the fusion of multispectral images in the Lαβ color space in [70]. It appears from this brief overview that measures based on wavelet representation or pyramid decomposition seem to be the most appropriate approach for image fusion. This is not only because of the inherent multi-scale aspect of the HVS but also because of the mathematical elegance and the possibility of signal decomposition for a more controlled and flexible fusion of the components at different levels of resolution. In that respect, we recommend the use of Bhatnagar et al. [70] for image fusion or any variants of the contrast measure defined in Eq. (74).

D. SEGMENTATION
Segmenting a structured image is rather an ill-posed problem due to the absence of well-defined and acknowledged objective criteria to define what could be considered as a good segmentation output. However, as it is well-known, the visual system is one of the best systems for segmentation and simplification of observed scenes. Several researchers have focused their approach towards exploiting knowledge about the mechanisms of the visual system in the development of efficient image segmentation methods [25], [26], [87], [159], [160]. For instance, the gray level thresholding method of Kundu and Pal [25] is based on the Weber-Fechner contrast measure. An interesting image segmentation method based on a visual nonlinearity model and Otsu gray level thresholding has been proposed in [26]. In this method a relative brightness difference measure, interpreted as a perceptual contrast, is defined. Another region based image segmentation using the Weber-Fechner contrast and a homogeneity measure has been proposed in [160]. We should point out that in contrast to the other perceptual approaches for image processing, in these methods both the Devries-Rose region and Weber region ( Fig. 1(b)) are taken into account [26], [160]. Another method that combines both the Moon and Spencer contrast model and perceptual channel decomposition using the Gabor filter has been proposed in [63] for contour detection. This method is inspired by Burgi and Pun [87] method where other temporal and perceptual aspects are used in the design of image segmentation. The contrast defined in Eq. (62) is derived from the isotropic filter model used in the Burgi and Pun method and adapted from [91]. The use of a contrast measure integrating the relative change of luminance, spatial frequency and multi-scale aspects in the design of image segmentation is the most efficient way for segmenting images into meaningful components.

E. PERCEPTUAL CODING AND QUANTIZATION
The efficiency of any lossy image coding method is highly related to the perceptual quality of the reconstructed signal (decoded signal) [161], [162]. The visibility of artifacts that may result from the lossy compression are related to the contrast of the image. Through literature we have seen that the perceptual coding methods are mainly based on JND which is in fact related to the perceptual contrast [163], [164]. Therefore, in this section we briefly discuss some representative JND-based perceptual coding methods. The main idea behind the use of JND models for perceptual image coding is to control the distortions that may result from the different preprocessing of the input signals and specially quantization effects [58], [165], [166]. Although the Weber-Fechner contrast measurement is not suitable for natural and complex images, it has often been used to optimize the image signal quantization process. Indeed, very often the perceptual quantization criteria and schemes used are mainly based on the Weber-Fechner contrast measure and the CSF [167], [168]. One of the pioneer works on the use of a more elaborated contrast model, and particularly the Moon and Spencer Model Eq. (7), for optimizing the quantization of picture signal has been reported in [36].
One of the challenging problems in perceptual coding is to achieve the compromise between the bit rate and distortion. Unfortunately, the JND models are not mathematically tractable in the Rate Distortion (RD) mechanisms. Nevertheless, there have been attempts made to simplify the JND for RD optimization in lossy image compression methods.
Another important aspect that has been investigated in perceptual coding is the use of visual masking models to predict the visibility of distortion due to quantization. In one study a new approach to directly measure masking in natural images was introduced [169]. In another study the visibility of distortion is estimated using a masking model based on a non-linear process involving the contrast of the masking signal and the threshold contrast [170]. The perceptual coding scheme proposed in [171] makes use of the Watson-Solomon Contrast Gain Control (CGC) [115] model to optimize the performance of High Efficiency Video Coding (HEVC) technique. It is worth noticing that most of visually lossless coding methods in the literature exploit the contrast measure in an explicit or implicit manner [172]. The only contrast model for perceptual coding and quantization that seems to be complete and efficient is the one introduced recently in [171]. Therefore, we recommend using this model for perceptual coding. From this brief overview it can be noticed that perceptual image compression mainly relies on the concept of CSF, JND, and visual masking. However, very few methods use explicitly a perceptual contrast measure. For example, the perceptual irrelevancy reduction scheme that can be considered as a perceptual compression, proposed in [33] explicitly uses a perceptual contrast measure based on the Peli and Moon and Spencer contrasts.

F. PERCEPTUAL WATERMARKING
Watermarking is the process of inserting the message (watermark) in the host signal in a way that it is not visible. In case of digital images, the challenge is to find a trade off between the robustness against attacks and transparency. In other words, robustness requires more energy to put in the watermark to make it less prone from the attacks but at the expense of more visibility of the watermark and hence losing the transparency. This is why the research community has paid more attention to perceptual watermarking approaches. In such an approach, image contrast is then a key factor for controlling the visibility of the watermark to the observer [173]- [175].
The notion of JND is highly linked to the perceptual contrast and is widely used to modulate the watermark signal. The idea is to control the visibility of watermark by exploiting the visual masking phenomenon which involve using models of local contrast map. This idea has been exploited in various perceptual watermarking methods particularly in [176] such as the use of the wavelet based contrast as defined in Eq. (48).
Another interesting perceptual watermarking method based on digital adaptation of the Moon and Spencer contrast [32] and as defined in Eq. (7) has been proposed in [35]. Following the same idea, a robust perceptual watermarking approach based on the concept of perceptual pyramidal decomposition and a JND model has been developed in [34], [177]. It is worth noticing that the notion of JND has been more exploited in the design of perceptual watermarking methods. Very few watermarking techniques use explicitly the perceptual contrast. However, to the best of our knowledge there is no model on the multi-scale JND. Whereas, several multi-scale models for the perceptual contrast have been proposed in the literature. Furthermore, the contrast not only contains implicitly the JND value but contains more information on the image signal. It is therefore, recommendable to use the contrast in the watermarking instead of the JND. Any of the wavelet-based contrast models Eq. (74), Eq. (48), or the multi-scale adaptation of the contrast defined in Eq. (50) is a good candidate for efficient perceptual watermarking.

G. OTHER POTENTIAL APPLICATIONS
Apart from these applications where the use of visual contrast is quite explicit and plays a predominant role, there are other areas where the notion of contrast, somewhat different from that well established in the field of psycho-physics and neurosciences, is used. These include, for example, fields of target detection [178], traffic road visibility [179], [180], face recognition (through LBP) [106], image classification [181] and other computer vision applications. In a method combining LPB descriptors with a local contrast measure for face recognition has been proposed in [107]. In this method the local variance of pixel values is considered as a local contrast measure. Another interesting application of contrast notion for road visibility estimation, under bad weather conditions, based on a neural network has been proposed in [182]. Note that, among the contrasts discussed in this study, the GCF [48] contrast seems to be the most suitable measure, in terms of simplicity and efficiency, that could be used in various interesting applications such as content based image retrieval, visualization and tone mapping techniques as suggested in [48]. Another interesting application where the contrast is of great importance is in the automotive imaging technology. A new concept called Contrast Detection Probability (CDP) has been introduced recently as a potential approach to analyse the performance of imaging systems in order to improve the contrast sensitivity in some difficult contexts such as automotive imaging and especially HDRI [183], [184]. It should be noted that in these very recent studies the Weber-Fechner contrast is used as the main attribute in the implementation of the CDP model. Of course, we cannot ignore the importance of discussing the notion of contrast in the current context of the deep neural network (DNN) revolution. Indeed, it is interesting to study the notion of contrast in the understanding and improvement of DNN based models for classifying objects. In a very recent study, the authors explored the concept of contrast in the context of DNNs [181]. This study focused more specifically on the effect of local and global contrast variation on the performance of object classification in an image. It has been shown that the decrease in contrast at randomly selected points significantly affects DNNs performance. It was also shown that reducing the contrast of pixels not aligned along contours further degrades the performance of object classification process based on DNNs. We believe that exploring through experimental and theoretical studies the links between DNN and some of the low-level features of the HVS, and especially perceptual contrast measure, would open promising approaches for solving complex problems in computer vision.

VI. CONCLUDING REMARKS AND FUTURE CHALLENGES
Through this study, it is clear that the concept of contrast is still a hot topic not only in vision research but also its use in the development of efficient methods for visual information processing and analysis. One can conclude from this critical study that despite numerous theoretical and experimental studies dedicated to contrast definitions and analysis, it is still difficult to have a clear picture on this very important feature of the HVS. For example, there is no criteria on how to compare the different contrast maps corresponding to different contrast definitions. Moreover, there is no proper definition that can completely take into account the human visual perception and overcome the limitations of the existing contrast measures.
Lessons learnt from this critical study could be beneficial for developing various methods for low-level (image registration, quantization, enhancement, etc.) and high-level (recognition, tracking, IQA, etc.) vision tasks. In this study we provide examples and guidelines on how different contrast measures should be used for various types of applications including but not limited to IQA, image quality enhancement, image fusion, image segmentation, perceptual coding and quantization, perceptual watermarking, target and face detection, road visibility and autonomous driving, image classification, etc. As an example, in the case of applications such as the development of image and video processing algorithms, a simple and mathematically tractable contrast measure in the parameter optimization of the methods could be sufficient.
The approach that seems the most pragmatic is to take inspiration from models of perceptual contrast measurement derived from psycho-visual experiences. This could be achieved by extracting only the most relevant and simplest aspects to be used in the definition of contrast for its direct use in the processing and analysis of digital images. Therefore, a reliable contrast measure should take into account several relevant factors related to the human visual perception. Among these factors and aspects, the most important in the design of contrast measures are the spatial representation which conforms to the foveal image model, for example the Moon and Spencer model, frequency selectivity, directional selectivity, the multi-scale aspect, and color.
Besides these well studied characteristics and factors of contrast definition measurements, several challenges are still there to be addressed in the future. Among these the temporal aspect of the visual contrast is one of these challenges that has not been considered in any of the existing measures.
Another interesting and challenging issue which needs further investigation, is to develop a contrast measure taking into account the binocular aspects for processing and analyzing stereo-images. Despite intriguing work on binocular vision and contrast, there is no finding that could be used in the design of a practical contrast measure that could be used in the case of stereo-images for multi-view processing and coding. Indeed, the existing studies only focus on the psycho-visual and neuro-physiological aspects.
It is worth noticing that the color contrast definitions and measures introduced and discussed in this study are also limited and do not take into account the inter-channel interaction in the design of the contrast. To the best of our knowledge, there is no well documented study of the modeling of the JND and the contrast for the color stimuli. Note that the last stage of the visual signal processing pipeline in the HVS related to the fusion and detection processes is often described using empirical models or probabilistic formalism inspired by the signal detection theory. Therefore, it is extremely important to develop such models as the ultimate stage of the visual signal processing and analysis chain which is critical and decisive in many respects. Finally, with the increase in the use of DNNs in different image processing and computer vision application we believe that we should consider methods to link DNNs and the HVS. This is especially the case of low level features of the HVS such as the perceptual contrast in order to develop new methods to not only solve complex problems in computer vision but also to contribute to the new research focus towards explainable DNNs.
We hope that this article will help researchers working in this field in designing a new contrast measure that is more adapted to the HVS and use existing contrast for particular application efficiently. To facilitate this goal, we will provide source codes of different contrast measures proposed by our group (Image Quality Working Group) on our website: https://www.imagequality.eu/.