This paper proposes a novel approach to specific object detection in complex scenes using color-based entropy features and a fuzzy classifier (FC). Appearances of the detected objects are assumed to contain multiple colors in non-homogeneous distributions that make it difficult to detect these objects using shape features. The proposed detection approach consists of two filtering phases with two different novel color-based entropy features. The first phase filters a test pattern with the entropy of color component (ECC). A self-splitting clustering (SSC) algorithm is proposed to automatically generate clusters in the hue and saturation (HS) color space according to the composing pixels of an object. The ECC value is computed from histograms of pixels in the found clusters and is used to generate object candidates. The second filtering phase uses the entropies of geometric color distributions (EGCD) to filter the object candidates obtained from the first phase. An EGCD is computed for each of the clustered composing colors of a candidate object. The EGCD values are fed to an FC to enable advanced filtering. A new FC using the SSC algorithm and support vector machine (FC-SSCSVM) for antecedent and consequent parameter learning, respectively, is proposed to improve detection performance. Experimental results on the detection of different objects and comparisons with various detection approaches and classifiers verify the advantage of the proposed detection approach using the FC-SSCSVM.