Improving the Generalised Few-shot Learning by Semantic Information

Human beings can learn novel vision categories with a few images. To mimic the ability, few-shot learning task becomes a hot topic recently. But it seems that most prior works neglect the fact that we as humans will not forget our original knowledge after learning novel categories. To reach this goal, we investigate the generalised few-shot task. Besides, when we learn new vision categories, the semantic information is usually helpful to link the new categories with our prior knowledge. Therefore, we expect the life-long learning model that can also link to the textual semantic information as human beings do. In this paper, we propose a two-head model including visual learning and textual learning. In visual learning, we use a classifier weights generator to help recognise the novel categories without changing the pre-trained feature extractor and the classifier weights for the base categories. In the textual learning component, we construct a classifier for all the categories using graph convolutional neural networks with the help of the knowledge graph. Finally, we use a simple weighted fusion technique to combine them to get the final prediction. To prove the validation and effectiveness of our model, we conduct experiments on ImageNet-FS and MiniImagenet to show that our result surpasses the previous state-of-the-art methods in this setting.