Gathering Effective Information for Real-Time Material Recognition

Material recognition is a fundamental problem in the field of computer vision. Material recognition is still challenging because of varying camera perspectives, light conditions, and illuminations. Feature learning or feature engineering helps build an important foundation for effective material recognition. Most traditional and deep learning-based features usually point to the same or similar material semantics from diverse visual perspectives, indicating the implicit complementary information (or cross-modal semantics) among these heterogeneous features. However, only a few studies focus on mining the cross-modal semantics among heterogeneous image features, which can be used to boost the final recognition performance. To address this issue, we first improve the well-known multiset discriminant correlation analysis model to fully mine the cross-modal semantics among heterogeneous image features. Then, we propose a novel hierarchical multi-feature fusion (HMF
²
) model to gather effective information and create novel yet more effective and robust features. Finally, a general classifier is employed to train a new material recognition model. Experimental results demonstrate the simplicity, effectiveness, robustness, and efficiency of the HMF
²
model on two benchmark datasets. Furthermore, based on the HMF
²
model, we design an end-to-end online system for real-time material recognition.