Landmark detection with surprise saliency using convolutional neural networks

Landmarks can be used as a reference to enable people or robots to localize themselves or to navigate in their environment. Automatic definition and extraction of appropriate landmarks from the environment has proven to be a challenging task when pre-defined landmarks are not present. We propose a novel computational model of automatic landmark detection from a single image without any pre-defined landmark database. The hypothesis is that if an object looks abnormal due to its atypical scene context (what we call surprise saliency), it then may be considered as a good landmark because it is unique and easy to spot by different viewers (or the same viewer at different times). We leverage state-of-the-art algorithms based on convolutional neural networks to recognize scenes and objects. For each detected object, a surprise saliency score, a fusion of scene and object information, is calculated to determine if it is a good landmark. In order to evaluate the performance of the proposed model, we collected a landmark image dataset which consists of landmark images, as we have defined them with surprise saliency above, and non-landmark images. The experimental results show that our model achieves good performance in automatic landmark detection and automatic landmark image classification.