Ontology-based digital photo annotation using multi-source information

The number of digital photos in the personal computer is exploding. In an effective photo management system, photo annotation is the most challenging task. The current photo annotation and management systems suffer from two crucial problems. One is the expression of semantic knowledge; the other is the way of photo annotation. Aiming at the former problem, this paper proposes to utilize ontology to organize the domain knowledge and provide formal, explicit and conceptual annotation. Meanwhile, a dual-level semi-automatic annotation approach is also proposed to resolve the latter problem. The rough annotation layer provides preliminary annotation by automatically extracting some semantic concepts from photo titles/texts, time concepts from EXIF metadata, and photo classification concepts from the result of face detection algorithms. The accurate annotation layer provides more detailed annotation by allowing users to modify, delete and add the annotation information freely. An ontology based photo management system OntoAlbum is implemented in this paper. Experimental results show that the proposed approach is very effective and promising.