Recognizing Induced Emotions With Only One Feature: A Novel Color Histogram-Based System

Emotions can be evoked in humans by images. Previous reports on Recognition of Emotions induced by Visual Content of images (REVC) mainly focused on numerous features to improve recognition performance. To devise a more robust REVC system, this paper examines the performance of a wide range of classifiers using color histogram as a single feature. Different numbers of color histogram bins in both RGB (red, green, blue) and HSV (hue, saturation, value) color spaces are considered in the examination and the overall classification performance is compared across the bin sizes. This investigation shows that features are not the only important factors affecting the performance of REVC systems, but also the type of classifiers and their parameters. This study shows that the HSV color space is better suited than the RGB color space for REVC systems. This paper proposes a new optimization algorithm called Optimizing Parameters of Ensemble RUSboosted Tree (OPERT) to boost the performance of the REVC system. Furthermore, a novel REVC system called Color histogram with Optimized RUSboosted Tree (CORT) is introduced. It is shown that our method is simpler, faster, and more efficient than the state-of-the-art, while providing comparable recognition performance. The robustness of the CORT system is validated over three different image datasets.


I. INTRODUCTION
Currently there is a plethora of images available online. These images are searched and used for numerous reasons: advertising, marketing, education, personal albums, for example. Because there are many images to be searched, automated retrieval systems are highly valued. One aspect of automated image retrieval is Recognition of human Emotions induced by the Visual Contents (REVC) [1]. REVC systems can be advantageous to various content indexing and retrieval applications [2] when looking for images based on their emotional contents, but not facial. For example: 1. A user searching among a large number of images on the Internet or their storage device to find an image (e.g., image of nature) to induce happiness or calmness for their meditation.
2. A magazine editor can benefit from using REVC systems to find an appropriate image for the cover page of the magazine to induce a particular emotion in the readers (e.g., a scene of World War II to induce horror/fear).
The associate editor coordinating the review of this manuscript and approving it for publication was Ahmed Farouk. 3. A campaign for famine relief would benefit from an image that induces sadness, so that people understand the human situation, and excitement, so people are motivated to act immediately.
However, despite efforts in this area [3], [4], there are issues (including modelling connections between low-level attributes, objects and scenes with abstract concepts such as emotions) that have not yet been solved. These issues are the result of the fact that images can be understood at high-level semantics including the affective level and the cognitive level [2]. Therefore, the current challenge in REVC is specifically predicting emotions induced by images.
The following review of prior literature establishes why this paper investigates color histogram-based REVC systems. The topic of REVC is not to be confused with human facial emotion recognition, which addresses the recognition of human emotions based on facial images [5].

A. REVC LITERATURE REVIEW
In recent years, more studies have focused on the visual influence of images on the human emotions. The relationship VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see http://creativecommons.org/licenses/by/4.0/ between the human emotions and visual contents of images has been investigated by different researchers [1], [3], [4], [6].
The research work has mainly focused on features of images such as color, texture, and shapes, as well as how these features affect human emotions. For emotion recognition, low-level features such as color and texture [3], [7], [8] as well as high-level features including human faces [3] were extracted from images. Solli and Lenz defined color emotions as ''feelings evoked by either single colors or color combinations and expressed with semantic words such as warm, soft, and active'' [6]. Joshi et al. [9] reasoned that because human emotions were highly correlated with aesthetics, aesthetic features including compositions, emphasis, and depth of field could be used for emotion predictions. Zhao et al. [4] showed that Principlesof-Art-based Emotion Features (PAEF) were giving better results than hand-crafted features in terms of affective image classification.
Among REVC techniques representing the state-of-the-art, Machajdik and Hanbury [3] adopted 114 features from different feature categories such as color, texture, composition, and content, which are commonly used in art and psychology. Colorfulness, hue statistics, color histogram, color names, and Itten contrasts are examples of color features that were used in [3].
Machajdik and Hanbury [3] used two feature selection methods, wrapper-based method and single feature method; as well as, feature dimensionality reduction method, Principal Component Analysis (PCA), in their work to reduce the number of features. In order to obtain more reliable results, they used 5-fold cross-validation method. Their reported results were based on the best features for every emotion category of the datasets used in their work (such as Art Photo) [3]. Zhao et al. later proposed PAEF in [4] and reported that their achieved results outperformed those reported in [3] with 5% improvement in classification accuracy on average. Zhao et al. used 165 features from different principles including harmony, movement, and variety when working with Support Vector Machine (SVM) in conjunction with radial basis function (RBF) kernel to classify categorical emotions. PAEF was proposed because the authors claimed that they found a weak connection between emotion and EAEF (Elements of Art based low-level Emotion Features) [4]. Zhao et al. [4] employed PCA to reduce the feature dimensionality and ran 5-fold cross -validation method on their datasets over 10 runs [4]. The average of accuracy over 10 runs was reported and compared with the-state-of-the-art such as that reported by Machajdik and Hanbury in [3].
Previous reports in REVC were mainly focused on extracting and making use of different features to improve recognition performance of REVC systems [3], [4]. Using a large number of features, increases the computation cost and calls for using feature selection and feature dimensionality reduction methods which imposes additional computation burden on the REVC system. This investigation aims to find a key visual feature, which is more effective than using a combination of different range of features with having significant reduction in computation cost and comparable performance with the-state-of-the-art.
Since there is a connection between color and emotions in psychology research [10], [11], the hypothesis is that color histograms is a potential key feature for investigation. This paper investigates color histogram-based REVC system.
A color histogram represents the distribution of pixel colors across an image and is an important technique for color feature extraction and classification. Color histograms have been widely used in many applications such as object recognition and content-based image retrieval (CBIR) [12]. The number of bins in a color histogram is important because it affects the system performance as well as the speed of computation. When more bins are used in color histogram, more memory is required with an increase in computation costs.
A color histogram with 256 bins was used for image clustering by Malakar and Mukherjee [19] with an accuracy rate of 87% for a small set of 40 images. An RGB color histogram with 48 bins was used on 500 images, taken from SIMPLIcity, in a content-based image retrieval system described by Chakravarti and Meng [31]. However, there was no reason given for choosing 48 bins in their investigation.
Color histograms have been rarely used in REVC [1], [3] and in an emotion image retrieval system [6], which is closely related to REVC. For instance, a color histogram was used for a color emotion image retrieval system as reported by Solli and Lenz where 8 bins were chosen [6]. Color histograms with 2 bins, 7 bins and 10 bins were used in an REVC system reported by Machajdik and Hanbury [3] for different image datasets without mentioning the reason of choosing these number of bins in their investigations or even stating which was found to be best.
Two issues arise from a survey of literature with regard to the use of color histograms for REVC and emotion-based image retrieval systems. First, the bin size of the color histogram used in the reported experiments was not often provided (e.g., [3]). Second, if the number of bins chosen for the histogram was mentioned in the research, little or no reason was given behind the selections or decisions made in this regard (e.g., [3], [6]).
Color histograms have been rarely used by researchers in connection with REVC systems, and when they were used, there was no or little investigation about the effect of using different numbers of color histogram bins in reported experiments (e.g., [3], [6]). To the authors' best knowledge, only Mohseni et al. [1] investigated the performance of a color histogram-based REVC system based on overall performance of classifiers [1], but not for individual emotion categories. Color histogram was examined in both HSV and RGB color spaces and it was shown that 8 bins is the optimum number of bins in both RGB and HSV color spaces based on the overall performance of classifiers [1].
Nevertheless, there remain several critical questions unanswered, including i) What is the recognition performance of each classifier employed in an REVC system for each emotion category? ii) How does the number of color histogram bins affect the recognition performance in each emotion category? iii) How do REVC systems perform in dealing with emotion categories using different color spaces?
Whereas Machajdik and Hanbury [3] and Zhao et al. [4] focused on REVC systems using variety of features, and of high computation complexity, this research investigates the bin size, types of classifiers (with varying parameters), and their impact on REVC performance of a color histogram-based system in terms of recognition accuracy, efficiency in computation and memory usage.
This paper aims to propose a new method and introduce a simpler, faster, and more efficient solution with REVC performance better than the state-of-the-art. This is done by considering a number of factors that affect color histogrambased REVC system performance including bin size, color space, and machine-learning algorithm. To ensure that results are not limited to one classifier, twenty-two different classifiers with three standard emotion image datasets, Art-Photo [3], Abstract paintings [3] and IAPSa [32] are used for REVC systems.
The main contributions of this paper are as follows. The paper identifies the optimum number of bins in color histograms for both RGB and HSV color spaces as well as the most suitable classifiers for both color spaces and compares those for REVC systems. It also presents a novel optimization algorithm for the above systems. Using those results, a new and more efficient REVC system is proposed and thoroughly tested, validated and compared with the state-of-the-art methods.
The remainder of the paper is structured as follows: experimental setup is described in Section II. Section III presents results and analysis of the RGB and HSV color spaces. Optimization and the proposed OPERT (short for Optimizing Parameters of Ensemble RUSboosted Tree) method is explained in Section IV; and the proposed new REVC system, which is called Color histogram with Optimized RUSboosted Tree (CORT) with its validation is proposed in section V; finally, the conclusion is drawn in Section VI.

II. EXPERIMENTAL SETUP
The experiment design includes image database, choice of color histogram bins, classifiers, performance evaluation criteria, and optimization for REVC.

A. IMAGE DATASETS
Three standard emotion image datasets ArtPhoto [3], Abstract paintings [3], and IAPSa [32] were used to ensure that the results are not limited to one image dataset. These datasets have been widely used in affective analysis research [3], [4]. Because these image datasets are unbalanced across emotion categories (each category has a different number of images, as shown in Table 1) corrective action, which will be discussed later, will need to be taken.

1) ART PHOTO
ArtPhoto [3] is a set of 806 artistic photographs extracted from DevianArt [33] using the search engine, which uses emotion categories as search terms. Image emotion tags were specified by the respective professional photographer at upload. Each tag is one of the eight different emotion categories proposed by Mikels et al. [32]: Amusement, Anger, Awe, Contentment, Disgust, Excitement, Fear, and Sad [1], [3], [4], [32], [34]. An example from each of the eight emotion categories is shown in Fig. 1.

2) ABSTRACT PAINTINGS
Abstract paintings [3] was created to investigate the influence of certain image features such as color and texture on evoking emotions. It consists of 229 images refined from 280 after rating in a web-survey by approximately 230 people. Each image was rated approximately 14 times against the eight Mikel's emotion categories [32], mentioned earlier. Each image was labeled with the category receiving the most votes. Images with the same label were grouped as the respective ground truth. Images receiving equal votes were removed from the image dataset. An example from each of the eight emotion categories is shown in Fig.2.

3) IAPSA
IAPSa [32] is a subset of the International Affective Picture System (IAPS) and contains 395 images. IAPS is a standard emotion-evoking image dataset in psychology [35] and contains 1182 of documentary-style natural color images of different domains such as portraits, insects, nature, animals, and babies. Labeling images in IAPS is based on the empirically derived mean and standard deviation of valance, arousal, and dominance from 1 to 9. Unlike ArtPhoto and Abstract   paintings, emotional labels in IAPS are not categorical. Therefore, Mikels et al. [32] selected 395 pictures from IAPS and categorized them into the same eight emotion categories that are used in ArtPhoto and Abstract paintings image datasets using votes from independent participants [32]. An example from each of the eight emotion categories is shown in Fig.3. Table 1 shows the number of images in each emotion category.

B. IMAGE PRE-PROCESSING
All images were normalized to equal-sized RGB images of 200×200 pixels for two purposes. First, it saves memory that reduces computation cost with relatively small sized images without losing too much information. Second, this removes the need for normalization, and makes comparison of the results straightforward. Cubic interpolation [36], [37] and antialiasing methods were used during resizing to reduce lost information and aliasing, respectively. Because images are processed in two color spaces, equal-sized RGB images were converted to HSV color space [38] [39]. The same procedure, explained above for RGB, was used for the HSV color space.

C. FEATURE EXTRACTION
In order to extract features, the normalized color images (RGB and HSV) were split into three planes (red, green and blue) [1] and color histogram was applied to each plane (Fig.4). Therefore, the dimension of the extracted feature vector becomes 3 (because there are 3 color channels) times the number of bins. For example, the dimension of the extracted feature vector is 48 if 16 bins is selected. In this research, color histogram with different numbers of bins was applied to the color channels of images in RGB and HSV color spaces (Fig.4). Different numbers of bins considered in this research were from 8 different levels of power of base 2 (i.e. 2,4,8,16,32,64,128,256) since there are 256 different intensity levels available to describe color distribution across each of the red, green, and blue (hue, saturation and value) color channel.

D. CLASSIFIERS
Because performance of various learning algorithms may vary with different applications, to identify the best classifier for REVC systems and ensure robustness in the results, twenty-two widely used classifiers with specified parameters were used in this investigation and listed in Table 2 [1]. Features, in the form of a feature matrix, were fed into the 22 classifiers to perform emotion recognition tasks. The feature matrix was constructed from the features extracted from color histograms in either the RGB or the HSV color spaces (Fig.4). The extracted feature vector associated with each color channel in RGB is concatenated with the feature vectors from other color channels and then constructed into the RGB feature matrix. The same procedure was carried out for HSV (Fig.4). Each image dataset was split into training and test sets. The training set was used for training the classifiers and the test set was used to evaluate the performance of each classifier.
In order to avoid over fitting problems and having more reliable results, 5-fold cross-validation method [3], [4] was used to split the dataset into training and test sets. In 5-fold cross-validation, first, the dataset is divided into five segments where four segments are used for training and one segment is used for testing. In the second iteration, again, the dataset is divided into the same five parts, but this time four different segments (from the first iteration) are used for training and a different segment is used for testing. This procedure continues until the fifth iteration when all data have been used for training and testing. At the end, average of the results over the five iterations is calculated and reported. The advantage of using this method of data partitioning is that not a portion, but all the data in the dataset are used for training and for testing purposes.

E. PERFORMANCE EVALUATION
Different performance evaluation methods have been proposed for classification-based applications. Overall True Positive Rate (OTPR) has been used previously to evaluate the effectiveness of the employed classifiers in the HSV and RGB color spaces [1]. However, because the image datasets are unbalanced and performance evaluation is needed for each emotion category, performance evaluation methods, such as OTPR (which provides evaluation for the overall performance of classifiers, but not individual category) could not be used in this investigation.
There are eight different emotion categories in the image datasets, the number of images in each category is different, and this makes these image datasets become unbalanced.
In this study, in order to overcome the dataset unbalance problem, True Positive Rate per class Averaged over the positive and negative classes (TPRA, for short) was adopted from [3], [4] to evaluate performance effectiveness, as defined in (1). This performance evaluation is One vs All (One category against all).
TP is true positive, FN is false negative, TN is true negative, TPR is true positive rate, and TNR is true negative rate. Average of TPRA over eight emotion categories (ATPRA) is defined as follows  (5), is another metric that used in this this study to evaluate the performance of classifiers.
where B denotes the number of selected bins in an experiment. In this experiment B = 8. TPRA (2 3 ) denotes the TPRA value of color histogram with 2 b number of bins. For example, TPRA (2 3 ) shows TPRA value for 8 bins.

III. REVC PERFORMANCE EVALUATION OF CLASSIFIERS
This section evaluates the REVC performance of twenty-two classifiers with different color histogram bin sizes in the RGB and the HSV color spaces, respectively, with their parameters specified in Table 2.
To evaluate the performance of classifiers, two measures were taken: 1) The relative performance of each classifier against all other classifiers for each color histogram bin based on TPRA, as defined in (1), and ATPRA, as defined in (4).
2) Average of the performance of a classifier over different color histogram bins (BTPRA) as defined in (5).
It is noted that BTPRA measures the average performance of a classifier over different studied bins. A higher BTPRA means better performing classifiers.

A. REVC PERFORMANCE EVALUATION IN THE RGB COLOR SPACE
The performance of each classifier was computed at a particular color histogram bin in terms of TPRA (Table 3). For instance, TPRA value achieved by Complex tree classifier for 2 bins in the RGB for Amusement is 53.73%. Moreover, the average of TPRA across all number of bins (BTPRA), defined in (5), was computed for each emotion category ( Table 3 shows that BTPRA for Complex tree in RGB for Amusement is 52.62% and is the average of Complex tree performance across 2 bins to 256 bins). A similar procedure was carried out for other emotion categories and the averages of TPRA across all emotion categories (ATPRA) as defined in (4) were computed (Table 4). For example, the ATPRA value obtained by Complex tree classifier in the RGB over eight-emotion categories is 53.52% (Average of all TPRA values achieved by Complex tree at 2 bins over eight emotion categories). Table 4 shows that Ensemble RUSboosted Tree classifier (RUSboosted, for short) [40], [41] achieved the highest performance in the RGB color space in terms of ATPRA, defined in (4), in most of the bin sizes and BTPRA,defined in (5), compared with other classifiers ( Table 4). The parameters selected for RUSboosted in this investigation is (M = 20, K = 30) and learning rate of 0.1, where M is the maximum number of splits in a weak learner and K is the number of weak learners/number of learners).
In order to identify the optimum number of bins in the RGB color space, the average of ATPRA for each bin size over all emotion categories was computed (Table 4). Table 4 shows that 8 bins with 52.62% had better achievement than other number of bins in the RGB color histogram. As such, 8 bins is considered as the optimum number of bins in the RGB color histogram. This finding is consistent with the results reported in [1] where different performance evaluation method was used.
Since color histogram with 8 bins in the RGB color space enabled the RUSboosted to achieve the highest REVC performance with reduced computational cost, the performance of this novel REVC method over all the eight emotion categories was compared with that of the known state-of-the-art reported in [3], [4] in terms of TPRA (Fig. 9).
As shown in Fig.9, the method of using the RUSboosted with color histogram with 8 bins in the RGB color space achieved a comparable performance in some emotion categories to that of the state-of-the-art reported by Machajdik and Hanbury [3] using the ArtPhoto image dataset.

B. REVC PERFORMANCE EVALUATION IN THE HSV COLOR SPACE
The performance of twenty-two studied classifiers, in terms of TPRA, was evaluated across different bin sizes for all eight-emotion categories in the HSV color space. In order to compare the performance of classifiers, BTPRA, defined in (5), was calculated for each emotion category. Table 5 shows that BTPRA for Fine KNN in HSV for Amusement is 53.72% and is the average of Fine KNN performance across 2 bins to 256 bins).
A similar procedure was carried out for other emotion categories and the average of TPRA across all emotion categories (ATPRA), defined in (4), were computed (Table 6). For example, Table 6 shows that the ATPRA value obtained by Fine KNN classifier at 8 bins in the HSV over eight emotion categories is 54.60% (Average of all TPRA values achieved by Fine KNN at 8 bins over eight emotion categories). The Ensemble RUSboosted Tree classifier (RUSboosted, for short) [40], [41] achieved the highest performance in the HSV color space in terms of ATPRA, defined in (4), in all the bin sizes and BTPRA, defined in (5), compared with other classifiers (Table 6). RUSboosted was used with M = 20, K = 30 and learning rate of 0.1, where M is the maximum number of splits in a weak learner and K is the number of weak learners /number of learners) ( Table 2). Therefore, RUSboosted was identified as the best classifier among the twenty-two classifiers for REVC in the HSV color space.
In order to identify the optimum number of bins in the HSV color space, the average of ATPRA for each color histogram bin over all emotion categories was computed (Table 6). Table 6 shows that 8 bins with 52.97% had better achievement than other number of bins in the HSV color histogram. As such, 8 bins is the optimum number of bins in the HSV color histogram.
A color histogram with 8 bins in the HSV color space enabled the RUSboosted to achieve the highest REVC performance with reduced computational cost (Table 6). Therefore, the performance of this novel REVC method over all the eight emotion categories was compared with that of the known state-of-the-art reported in [3] and [4] as well as the results obtained in the RGB color space, in terms of TPRA (Fig. 9).
As shown in Fig.9, the method of using the RUSboosted with color histogram and 8 bins in the HSV color space achieved better performance than that of the state-of-theart reported by Machajdik and Hanbury [3] but weaker than that of reported by Zhao et al. [4] using the ArtPhoto image dataset. Therefore, optimization of the RUSboosted performance, which is explained in section IV.A, is needed. Moreover, it is clear from Fig. 9 that the HSV color space outperformed the RGB color space.

IV. OPTIMIZATION OF RUSBOOSTED CLASSIFIER A. OPTIMIZATION PROCEDURE
The aim of optimization is to maximize the performance of RUSboosted in terms of TPRA in REVC systems. In section IV.B, the proposed OPERT (Optimizing Parameters of Ensemble RUSboosted Tree) algorithm for optimization process is explained. The improved performance from optimization in both HSV and RGB color spaces in REVC system are analyzed in section V, respectively, and then compared with the-state-of-the-art.
As mentioned earlier, RUSboosted was discovered to be the best of the 22 studied classifiers when working with color histogram in both RGB and HSV color spaces. The parameters set for RUSboosted were M = 20, K = 30 with a learning rate of 0.1, where M is the maximum number of splits in a weak learner and K is the number of weak learners (number of learners).
An ensemble classifier is constituted of a constellation of different classifiers [41]. When these classifiers work together, they make a stronger classifier, which is so-called an ensemble classifier, and each contributing classifier is called a weak classifier (or a weak learner) because their performance is weaker than the performance of an ensemble classifier. The role of the number of weak learners in constructing a powerful ensemble classifier is undeniable. Therefore, the number of weak learners is an important factor in optimization. In order to improve the performance of RUSboosted, its parameters M and K ( Table 2) were selected for more investigation in working with color histogram with 8 bins in the HSV color space. The HSV color space was selected because according to the previous findings shown in section III; RUSboosted had better performance in the HSV color space than in the RGB for REVC systems.
In order to identify the more appropriately ordered pair (M , K ) which can boost the performance, different combinations of M and K were tried for the optimization process for RUSboosted. To do this, in each emotion category, M and K started from 5 to 200 in steps of 5 leading to a total number of 1,600 combinations for each emotion category, (40 × 40 = 1600).
The aim is to find the optimum pair of (M , K ) among all these 1,600 combinations to gain higher TPRA performance in each emotion category. Therefore, the proposed algorithm searches for the local maxima and the global maximum among 1,600 different combinations. Because of the nature of the learning algorithm, the achieved TPRA with the same M and K at each run in each emotion category is different from other runs.
To improve reliability, the proposed algorithm with the same M and K values were run 10 times [4]. The average of the 10 different TPRA, achieved from these 10 runs, were compared with the previous performance reported in section III (using color histogram with 8 bins in both RGB and HSV color space, (M = 20, K = 30) as well as those reported in [3] and [4]) (Fig. 9).
It should be noted that due to the nature of the learning method, the achieved results by another for each emotion category might be different from what is reported here.

B. THE OPERT ALGORITHM
The aim of the OPERT (Optimizing Parameters of Ensemble RUSboosted Tree) algorithm is to find the optimum M and K values to improve the output of RUSboosted in each emotion category.
This algorithm starts from an initial point which is (M = 5, K = 5) and other parameters such as S and N denoting step size and the total number of steps, respectively. In this paper, In other words, after 40 increments by one step of 5 in each direction for both M and K , 1600 (40 × 40) different points in each emotion category are evaluated in terms of TPRA (as shown in Fig.8) where 40 out of these 1600 points belong to local maxima. As can be seen from Fig.8, the local maxima VOLUME 8, 2020 are displayed by red dots on the OPERT map, while the global maximum is illustrated by the green dot (Fig.8).
To illustrate how the OPERT algorithm works, an example is provided. Fig.5 represents a simplified OPERT map for one of the emotions with nine rows and eight columns to illustrate how M and K values are selected by the OPERT algorithm to maximize the performance of RUSboosted. In this map, each point has three coordinates of which the first coordinate denotes M value and the second belongs to K and the third coordinate shows the achieved TPRA value at that point by RUSboosted.
The algorithm starts with initialization of I , J , and D sets in line 5 of Table 7. Where N denotes the total number of steps and D is set of the step numbers. I and J are sets to control the iterations of M and K in RUSboosted [40], [41], respectively. where In this paper S = 5 and N = 40 (M i = 5 × i, K j = 5 × j for i ∈ I and j ∈ J ). The OPERT algorithm searches the OPERT map row by row and retrieves and stores the first coordinate of the local maximum (M ) in vector V L as shown in (7). This means that the first row of the OPERT map (j = 1) is searched for local maxima and in this example two local maxima are detected in columns M = 15 and M = 35 (Fig.5). The OPERT algorithm selects the smaller M value, which is 15 because smaller M value creates less complexity in ensemble classifiers; thus, the possibility of occurring overfitting is lower. Therefore,  where Q[j, i]= Q T [i, j] (Q T is a transpose of Q). Following this, the second row of the OPERT map (j = 2) is searched, but this time only one local maximum is detected; thus, the first coordinate of this selected local maximum which is 10 (because (i = 2) × (S = 5)) is stored in the second row of V L (V L [2] = 10) in (7). This process is continued until all rows in the OPERT map are searched and the first coordinate of the selected local maxima is stored in V L . Rows in the OPERT map correspond with the rows in V L (indexed by j). Therefore, V L has the same number of rows as the OPERT map, which is 40; however, in this example the number of rows is 9 with only a single column (V L is a column vector). The vector V L with the stored values is represented in (8). The explained process above can be found in line 6 to 13 in Table 7.
A new OPERT map, called OPERT map 2, represents the selected local maxima stored in V L (Fig.6).
The OPERT counts the number of local maxima in each column of the OPERT map 2. Optimum M and K are selected from the column contains more local maxima (the best column). If there is more than one best column, with equal number of local maxima, the column with higher value of K is selected as the best column (column i = 6 or M = 30 in Fig.6). Fig.6) which stores only different M values, their highest corresponding K values, and their corresponding local maxima counts, respectively. All these stored values are based on V L , which can be represented by a mapping from V L to N through (9) to (13).
where for 1 ≤ d ≤ D and d is in ascending order (13) and function countf (vector == value) counts the total number of elements in the vector which has the value. Take  (14). Those M and K coordinates that have the highest count of local maxima (line 32 in Table 7), i.e., (20,40) and (30,45) as shown in (14) are selected. Then that coordinate point with the highest K value is selected as the optimum solution (line 33 and 34 in Table 7), i.e., M Optimum = 30 and K Optimum = 45, which are used in RUSboosted as optimum values. The complete OPERT algorithm is described in pseudocode in  Table 7.
The OPERT algorithm was run for every emotion category separately, but just for demonstration purpose, the achieved results from Amusement is shown (Fig.8). Although the global maximum is larger than the stored local maxima in that emotion category, the selected optimum M and K combination is the most robust (lowest variance) of the global maximum and the largest of the local maxima (The optimum might be a local maximum and not the global maximum). Since M and K started from 5 to 200 in steps of 5, 1600 different combinations were available in the emotion space (e.g., Amusement). Some of these points had been identified as local maxima and one of these local maxima had been identified as the global maximum (as illustrated in Fig.8 for Amusement, as an example).

V. THE PROPOSED NOVEL REVC SYSTEM
A more robust REVC system is needed; thus, a novel color histogram-based REVC system, which is called Color histogram with Optimized RUSboosted Tree (CORT), is proposed in section V.A. Section V.B shows the results from using CORT with the three different datasets (each set is split into training and testing subsets using 5-fold cross validation). Measuring significance of the improvement gained by using CORT against the-state-of-the-art is evaluated by ANOVA and discussed in the same section. Furthermore, section V.C presents the results from training the CORT system with ArtPhoto and testing with Abstract paintings and IAPSa. This provides further validation of the robustness of CORT. This method of validation (training on one dataset and tested on distinctively different datasets) has not, to the knowledge of the authors, been done by others.

A. CORT SYSTEM
Color histogram with Optimized RUSboosted Tree (CORT) system is a robust REVC system that benefits from using the proposed OPERT algorithm (section IV.B) and other findings on improved performance documented in the previous sections of this paper (e.g. the best color space and the optimum number of bins).
The CORT system, outlined in (Fig.7), utilizes the proposed OPERT algorithm, explained in section IV.B, to optimize the RUSboosted performance and uses the color histogram as a single feature with the identified optimum number of bins (8 bins) and the HSV color space (discussed in section III.B). The OPERT algorithm maximizes the performance of the CORT by finding the optimum (M , K ) in each emotion category for RUSboosted. The schematic diagram of the CORT system is shown in (Fig.7).
The diagram shows that results of the RUSboosted classifier in each iteration is stored in Evaluation unit until the OPERT algorithm finds the highest performance of RUSboosted by finding the (M Optimum , K Optimum ). The Evaluation unit memory is then erased and the best performance of the RUSboosted classifier (comparing the predicted emotion labels and the ground truth) is stored in the Evaluation unit and appears in the output of the CORT system as the final result.
To ensure that the performance of the CORT is not only limited to one image dataset, its performance is evaluated over the three different datasets that were covered in section II.A.

B. RESULTS OF THE CORT SYSTEM 1) RESULTS OF THE CORT USING ARTPHOTO
The proposed CORT system was used with ArtPhoto image dataset and its performance across all emotion categories became statistically significantly superior to that reported by Machajdik and Hanbury in [3] and comparable to that reported by Zhao et al. [4]. While the CORT system is working with just a single feature, Machajdik and Hanbury [3] used 114 features in their experiment and Zhao's method [4] works with 165 different features. Additionally, because the proposed CORT system uses only a single feature (selected from the findings in psychology research [10], [11] that there is a strong connection between human emotions and color), it does not need any algorithm to reduce the number of features. This contrasts with the-state-of-the-art [3], [4], which used various feature selection methods and/or feature dimensionality reduction methods; imposing an additional computational burden. This shows the advantage of the CORT system, which is its simplicity by having a single feature, higher speed by having one feature and less computation burden, and accuracy as shown by performance.
The performance comparison shown in Fig.9 using Art-Photo dataset highlights the importance of both classifiers and optimization techniques.
The comparison between the proposed CORT system before and after using the OPERT algorithm in this paper shows that there is up to 6% improvement (Fig.9). In this investigation, the detected optimum (M , K ) for each emotion The found (M Optimum , K Optimum ) through OPERT could be the global maximum or a local maximum. For example, although the global maximum obtained a slightly better result than the optimal local maximum in Amusement (Fig.8), the local maximum is more robust since its variance around the mean (over 10 runs) is lower. As a result, (M Optimum , K Optimum ) in Amusement belongs to the identified local maxima, not the global maximum (Fig.8). It is noted, as mentioned earlier, that due to the nature of the learning methods, the achieved results by another for each emotion category might be different from what we have reported.
In addition, the CORT system was used for all emotion categories over different numbers of splits and weak learners and its performance was improved by the OPERT algorithm. The performance of the CORT system, even with RGB color space, was still superior to that reported by Machajdik and Hanbury [3] in all emotion categories. The CORT system with the HSV color space remained superior to when RGB was used. VOLUME 8, 2020

2) RESULTS OF THE CORT USING ABSTRACT PAINTINGS
To determine if the identified optimum (M Optimum , K Optimum ) for each emotion category obtained from section V.B.1 (Art-Photo) is applicable to other image datasets, such as Abstract paintings, the CORT system with OPERT disabled was used with Abstract paintings. This means that the OPERT was disabled in the CORT system (CORT in the deactive mode) because we did not use OPERT to find (M Optimum , K Optimum ) for each emotion category in Abstract paintings for RUSboosted in the CORT; instead, we used the same identified (M Optimum , K Optimum ) for each emotion category obtained from section V.B.1 (ArtPhoto). If (M Optimum , K Optimum ) are applicable to other datasets, there should be no significant change in performance compared to the-state-of-the-art.
It is clear from (Fig.10) that the performance of the CORT system in the deactive mode across all emotion categories is significantly superior to that reported by Machajdik and Hanbury in [3] and better than that reported by Zhao et al. [4]. While the CORT system works with only a single feature, it should be remembered, as mentioned earlier, that Machajdik and Hanbury [3] and Zhao et al. [4] used multiple features and using various feature selection and feature dimensionality reduction methods. Therefore, the CORT system is again advantageous over the-state-of-the-art.
The results above support the hypothesis that the identified (M Optimum , K Optimum ) obtained from ArtPhoto (section V.B.1) can be used with other image datasets such as Abstract paintings. As a result, there appears to be no need to use the OPERT algorithm when (M Optimum , K Optimum ) from other image datasets is available.

3) RESULTS OF THE CORT USING IAPSA
Similar to what carried out in section V.B.2 with Abstract paintings, the CORT system was used in the deactive mode with IAPSa as well. This means that the OPERT was disabled in the CORT system and the identified (M Optimum , K Optimum ) for each emotion category, obtained from section V.B.1 (Art-Photo), was used with RUSboosted in CORT. Fig.11 shows that the TPRA performance of the CORT system across all emotion categories are comparable to that reported by Machajdik and Hanbury in [3] and Zhao et al. in [4]. While the CORT system works with only a single feature, Machajdik and Hanbury [3] and Zhao et al. [4] used 114 and 165 features in their experiment, respectively. Further confirming the validity of the proposed single feature system.
Once again, the same (M Optimum , K Optimum ) found using the OPERT algorithm with ArtPhoto for each emotion category, section V.B.1, works for another dataset: IAPSa. The one being Abstract Paintings, section V.B.2. It is again demonstrated that (M Optimum , K Optimum ) for each emotion category needs to be found only once using OPERT.
The results achieved by the CORT for the different image datasets revealed that the improvements made by the CORT in ArtPhoto and Abstract paintings were better than in IAPSa. While the performance of the CORT system in ArtPhoto and Abstract paintings are better than the-state-of-the-art, the performance of the CORT system in IAPSa is only comparable to the-state-of-the-art. Two theorized reasons for such differences are: 1) IAPSa was constructed at least 10 years before ArtPhoto and Abstract paintings. Thus, the technology used in taking photos of IAPSa is older, and color might not have been captured as precisely as for ArtPhoto and Abstract paintings.
2) The images of ArtPhoto and Abstract paintings are artistic, whereas the images of IAPSa are of normal daily. As such, artists might need to rely more on color to induce emotions, whereas documentary photographers rely more on objects/scenes.

4) STATISTICAL EVALUATION OF RESULTS
An Analysis of Variance (ANOVA) test [42], [43] was employed to reveal whether the achieved results by by CORT is significantly better than the-state-of-the-art or not. The null hypothesis is that there is no significant difference between this investigation and the state-of-the-art reported in [3] and [4].
To check whether the null hypothesis needs to be accepted or not, one-way ANOVA test [42], [43] was used to investigate the significance level among the reported results. Any p-value is less than 0.05 considered as a significant difference.
A one-way ANOVA test revealed that the proposed REVC system in this investigation, the CORT, is significantly better than Machajdik and Hanbury [3], using more than 100 features, in both ArtPhoto and Abstract paintings with significance level of p = 0.025 and p = 0.005, respectively (Table 8).
This means that the null hypothesis is rejected, and the alternative hypothesis is accepted in favor of the CORT system (HSV color space) because p <0.05. One-way ANOVA also showed that there is no significant difference between the proposed method by Zhao et al. [4] and the proposed method in this investigation. Therefore, it is shown that the achieved result in this investigation is comparable to that reported by Zhao et al. [4], using 165 features, over all the studied image datasets, but with lower computation cost, less memory usage, and easier implementation. Therefore, even though the CORT system uses only a single feature, the above statistical test (ANOVA) showed that the CORT performance is better or comparable with the-state-of-the art.

C. CORT SYSTEM VALIDATION
Although the benchmarks, Machajdik and Hanbury [3] and Zhao et al. [4] had not validated their results with different image datasets, this research validated the trained model with different image datasets.
For this validation, the trained model is the CORT system trained with ArtPhoto with (M Optimum , K Optimum ) for each emotion category, found in section V.B.1 (ArtPhoto). To avoid any confusion and for simplicity, this trained model is named as TCMAP (short for Trained CORT Model by ArtPhoto for Prediction).
TCMAP was used to predict emotions induced by images in Abstract paintings and IAPSa. Images in Abstract paintings and IAPSa were not given to TCMAP; thus, this method is a superior evaluation of model robustness.
All 229 images of Abstract paintings without their labels were given to TCMAP to predict emotions induced. The results are presented in (Fig.10). The same procedure was used with IAPSa and the result of prediction by TCMAP illustrated in (Fig.11).
From ANOVA, the performance of the TCMAP found to be significantly superior to that reported by Machajdik and Hanbury in [3] and better than that reported by Zhao et al. [4] ( Table 8).
This shows that TCMAP was well trained by ArtPhoto and it is ready to predict new given images from any image dataset. Recall, TCMAP uses one feature, whereas Machajdik and Hanbury [3] Zhao et al. [4] used 114 and 165 different complex features, respectively. This verifies the advantage of the proposed CORT system (TCMAP being a CORT system), which is its simplicity, higher speed, and accuracy.
The speed of the proposed CORT system (TCMAP) to predict images of IAPSa, as an example, was tested in Matlab and 395 images of this image dataset were processed in about 198 seconds (around 3.3 minutes) in computer system with AMD A8-7600 Radeon 3.1 GHz and 12 GB RAM, while it used, on average, about 4.8 MB memory per image. Fig.11 shows that the performance of the TCMAP in IAPSa is comparable to that reported by Machajdik and Hanbury in [3] and mostly worse than that reported by Zhao et al. [4]. However, ANOVA found no significant difference. The theorized reasons for this lower performance in IAPSa are those that were covered in section V.B.3.

VI. CONCLUSION
This paper has investigated and presented a novel method for recognition of human emotions induced by visual contents using the CORT system with the proposed optimization algorithm, the OPERT. It has been shown that designing sophisticated features based on art, psychology, or using a higher number of features does not necessarily lead to better REVC performance than fewer effective features, such as color histogram with 8 bins.
The performance of the proposed CORT method (as shown in this paper and by using a one-way ANOVA test) is significantly better than or comparable with the state-of-theart methods reported in the literature. Further, it has the added advantage of reduced computational cost and memory requirements.
The CORT system, using only a single feature, works better than the state-of-the-art in terms of efficiency, speed and simplicity, using numerous and complex features, across two of the studied image datasets and is comparable in the other image dataset.
The robustness of the CORT system was validated by using TCMAP over three different image datasets and the achieved results were predominantly positive.
This paper has shown that using mid-level features (such as PAEF proposed by Zhao et al. [4]) has no particular advantage over using low-level features such as color histogram in REVC systems.
This paper also demonstrated that REVC performs better in the HSV color space than in the RGB color space with respect to different emotion categories which is consistent with what was previously reported in [1] where a different performance evaluation method was applied. Therefore, it is recommended that HSV be the color space of choice for REVC systems.
In addition, it was shown that among twenty-two studied classifiers, Ensemble RUSboosted tree classifier was the best classifier for REVC system in both RGB and HSV color spaces. This investigation has changed our understanding about using the type of features (such as low-level features versus mid-level features) and using the number of features versus key-visual features) in REVC systems.