How to represent scenes for classification?

Object-based scene image representations can effectively capture the semantic meanings of a scene. However, they usually neglect a scene's structure information. In this paper, we propose a novel and effective detector-based scene representation method for scene classification. In particular, we extract object features by object detectors. By sensible principal component analysis, we obtain a compact representation vector of objects in a scene image. To capture the scene layout, we then train lots of deformable part models to form a scene response vector. By concatenating these two vectors we use a linear support vector machine for scene classification. When combining with DeCAF [1] in a special way, our method is even more powerful on complex scene categorization. Experimental results on the MIT indoor database show that our approach achieves state-of-the-art performance on scene classification compared with several popular methods.