Color Feature Based Dominant Color Extraction

The dominant colors in an image can be used for image search, color editing, palette generation, and several other applications. Conventionally, dominant colors are extracted using clustering or histogram-based methods. However, these methods cannot extract the dominant colors of small regions, which are essential for the analysis of color schemes. This study developed an approach to automatically extract dominant colors based on color features that are typically considered by human observers when analyzing color schemes. The proposed method first calculates the initial dominant color candidates using the K-means algorithm in the CIELAB color space and the graph cut of a region adjacency graph (RAG) of the segmented image. Next, the algorithm calculates the color features such as the saturation, contrast, and area of each cluster, based on which it extracts the dominant colors. Our method can extract prominent colors from small image regions as the dominant colors, which is not possible using conventional methods.

an image, colors that stand out despite occupying a small 23 area must be considered. For example, Fig. 1 shows an image 24 that is almost completely black or gray, with a small ochre- 25 colored moon. When analyzing this image, histogram-based 26 or clustering-based methods select only grays with various 27 brightness levels as the dominant colors. However, the con-28 trast between the black background and ochre region in this 29 image is essential. 30 This study proposes a new dominant color extraction 31 method to solve this problem. In this method, we first analyze 32 The associate editor coordinating the review of this manuscript and approving it for publication was Senthil Kumar .   36 Color schemes are a staple of research in the field of design. 37 Numerous books [11], [12], [13] and online services [14], 38 [15] provide examples of color schemes in images, designs, 39 and paintings. However, these sources provide only short 40 descriptions of terms such as ''base colors'' and ''accent 41 colors''. Quantitative descriptions of how to extract dominant 42 colors are not available.

II. PREVIOUS WORKS
This method first quantizes image colors using the gener- to obtain a stable initial cluster center with various colors.

56
However, this method occasionally fails to select conspicuous 57 colors limited to small regions. Furthermore, Chang et al. 58 proposed a method [17] to extract a compact set of domi-59 nant colors using the K-means algorithm and the basic color 60 categories [18]. In this method, a large number of initial to a single color layer. Therefore, the colors of the given 70 image can be manipulated by changing the palette colors.

71
The easiest method to obtain the palette color is the K-means 72 algorithm in the RGB color space [19]. Certain studies use 73 a convex hull that encloses all colors in the image and 74 regard the vertices of the convex hull as the candidate palette 75 colors. Subsequently, the final palette colors are acquired 76 by simplifying the convex hull [20]. In convex hull-based 77 approaches, colors tend to be saturated.

79
In this section, a dominant color extraction method is 80 explained.

82
To devise an effective dominant color extraction method, 83 we analyzed more than 500 data sets of color palettes from 84 images extracted manually. Based on this analysis, the fol-85 lowing hypotheses are developed: 86 1) The number of dominant colors in most cases ranges 87 from 3 to 6.
88 2) Colors with high saturation are often selected. In par-89 ticular, the most vivid color is almost always selected. 90 3) The color occupying the largest area is almost always 91 selected regardless of its conspicuousness.  5) The color that is significantly different from its sur-95 roundings is more likely to be selected.

96
To obtain an algorithm that extracts the dominant colors 97 based on the above features, the following steps must be 98 followed, resulting in the extraction of a 5-color palette.     The algorithm first applies the bilateral filter [21] to the 116 image for noise reduction. The filtered pixel value g(i, j) is 117 calculated as shown in (1).
is the pixel value of (i, j) in the CIELAB color 123 space, w is the kernel size, σ 1 and σ 2 are the smoothing param-

139
The three separated wall areas, cyan, brown, and magenta in 140 Fig. 3(b), are combined into one region. respectively. In Fig. 4(b), each pixel is painted with its 145 FIGURE 4. Initial dominant color candidates. associated cluster color that is one of the initial candidate 146 colors. Hereafter, the term ''candidate color'' k is used syn-147 onymously with ''cluster'' k.

149
In this section, we calculate the color contrast, area occupied 150 in the image, and saturation of each candidate color. The color 151 contrast is calculated using Itti's attention map [22]. In this 152 method, the hue, luminance, and edge direction contrasts are 153 calculated at various image scales to obtain both local and 154 global image contrasts.

155
The contrast C k of each candidate color k is defined in (2). 156 where N k is the number of pixels belonging to the cluster 158 k, and Fig. 5(a) shows an example of Itti's original 160 method applied to Fig. 4(a). The brighter pixels have a higher 161 contrast. Fig. 5(b) shows each cluster contrast by painting 162 pixel (i, j) according to the value of C k (pixel (i, j) ∈ k). 163 Following the color contrast, the area occupied by each 164 dominant color candidate is calculated by counting the num-165 ber of pixels belonging to cluster k. The area A k occupied by 166 candidate color k is defined as shown in (3).

167
A k = the number of pixels belonging to cluster k (3) 168 Then each A k is normalized using the maximum value of A 169 for all candidate colors.

170
In addition, the saturation S k of a candidate color k is 171 defined as shown in (4).  extraction. Fig. 6 shows an example.

188
The image on the left is the input image, which is partially  Fig. 4(a). Black represents 0, and white represents 1.

FIGURE 9. Final dominant colors for
The mask image m 0 is the image in which the pixels 206 belonging to cluster 0 corresponding to the blue sky in 207 Fig. 4(a) are painted in white, whereas the mask image 208 m 1 shows the pixels belonging to cluster 1 corresponding to 209 the boundary between the yellow flower and the blue sky are 210 painted in white. Cluster 1 is generated by mixing the yellow 211 flower and blue sky. It appears between the color boundaries 212 and are very narrow. Erosion is then applied to all the mask 213 images m k . If the white pixels disappear after the erosion, it 214 seems that the candidate color k appears due to color mixture, 215 and our algorithm deletes it from the candidate color list. Whenever we choose a candidate color as the final dom-227 inant color d, we multiply the weights w(d, k) to all feature 228 values of candidate colors k, i.e., S k , A k , C k and p k . w(d, k) 229 is defined as shown in (7).  Fig. 9 shows the final dominant colors for Fig. 4(a). The 242 leftmost yellow color is extracted because its saturation is the 243 highest among the candidate colors (Fig. 8). The second blue 244 color is extracted because its area is the largest. The remaining 245 colors are extracted based on the p k values.

247
We downloaded 300 images from the Internet and extracted 248 the dominant colors using our method alongside the methods 249 reported in previous studies. Fig. 10 shows the results. Each 250  Fig. 10(a)-(d) demonstrate that our method can extract 258 prominent colors that occupy only a small area. In Fig. 10(a), 259 the mark on the notebook is prominent. It comprises a thin 260 cluster of deep oranges and a small region of turquoise blue.

261
As expected, US succeeds in extracting them. In Fig. 10(b), 262 the contrast between red-eye and green leaves is dominant.

263
As expected, US can capture this feature. In Fig. 10(c), the 264 small yellow region is important for analyzing the color 265 scheme. Again, US can capture this. In Fig. 10(d), yellow,        In the experiment, an EIZO ColorEdge CG279X, a 27 inch 311 sRGB calibrated monitor, was used. The subjects positioned 312 themselves in front of the monitor at a distance of 100 cm. 313 In the monitor, the left side displayed the image, and the right 314 side displayed five types of extracted color palettes as shown 315 in Fig. 11. The order of the color palettes was shuffled for 316 each image.

317
The directions provided to the subjects are as follows.     Six Japanese participants in their 20s participated in this 330 experiment. The Ishihara test plate was used to confirm that 331 all participants have a normal color vision [26]. Each partic-332 ipant took approximately 20 min. to assess 23 images, i.e., 333 half of the entire dataset (46 images). In other words, each 334 image was tested by 3 participants. Table 1 shows the total scores obtained using each method. 336 These scores show that PI and US produced the best results. 337 Fig. 12(a) shows the case where the US score was bet-338 ter than that of PI. As shown in this example, US (the 339 proposed method) extracts small areas that include promi-340 nent colors. On the other hand, Fig. 12(b) shows a case 341 where PI performs better than US. The lower US score is 342 attributed to the same reason as described in Fig. 10(l). 343 This image comprises large achromatic regions with small 344 red regions. Consequently, S k and C k become large in the 345 reddish regions. Therefore, our algorithm produces multiple 346 reddish dominant colors. This problem should be addressed in 347 future research. features. Furthermore, we plan to apply our method to achieve 359 an automatic color scheme analysis.