Multimodal Prototype Enhanced Few-Shot Image Recognition with Dense Attention-Weighted Knowledge Graph Propagation

Few-shot learning aims to achieve precise novel category identification given limited labeled samples per class. Enhancing category representation with multimodal feature is a valuable scheme to compensate for the lack of labeled data. Knowledge graph is an easily accessible source of semantic knowledge. However, existing WordNet-based graphs contain noise interference and structural defects, which greatly affect the quality of semantic structure. Furthermore, there is a dilution of knowledge about distant nodes in graph propagation. Motivated by this, a multimodal feature enhanced few-shot image recognition approach, which combines prior knowledge from knowledge graph and visual prototype is proposed in this study. Class-level semantic information is utilized to validate and refine the knowledge graph to eliminate interference. Additionally, a dense attention-weighted knowledge graph propagation strategy is designed to discover more potential information. Built on semantic knowledge inference and integrated with visual feature learning, the proposed method constructs a more stable prototype representation to complete few-shot image recognition tasks. Comprehensive experiments conducted on miniImageNet and tieredImageNet benchmarks demonstrate that the proposed approach achieves favorable few-shot recognition performance compared with state-of-the-art methods.