For solving "semantic gap" which exists between the low-features and the high-level semantic features and the fuzziness of users' comprehension, combining Ontology and Fuzzy sets, an universal image semantic description model based on fuzzy domain ontology (SDMFDO) is constructed. Ontology is a kind of model that is used to describe the concepts and the relations of them, and fuzzy set theory can make image retrieval apart from precision of calculating. By adding fuzzy membership to the concepts and the relations of them in the domain ontology, we get a fuzzy domain ontology (FDO) which can be used to describe the semantic features of an image in a way catering for human's fuzzy thoughts. Then the mapping from low-features to high-level semantic features realizes using FSVMs. Finally, the simulation experiments on images provided by Corel are performed to confirm that this method has effectively improved the retrieval performance.