Recognizing Material of a Covered Object: A Case Study With Graffiti

Recognizing materials using image analysis is a classic problem. However, little research has been done with the images which have visual impediments such as noise, obstacle, or painting. This paper introduces the problem of recognizing covered materials which are distorted visually (e.g., materials covered by graffiti). We propose a set of approaches to solve this problem using a class of deep learning and transfer learning models, and evaluate our approaches empirically using a large-scale real world dataset that displays street scenes containing various materials which are covered with graffiti. Our experiments show that recognizing covered materials using the state-of-the-art approach for material recognition produced an mAP of 19%, while our proposed approach achieved an mAP of 60%. This evidently demonstrates that an approach for plain material recognition is not suitable for recognizing covered materials; hence this problem should be treated differently as in our proposed approaches.