Few Shot Model based on Weight Imprinting with Multiple Projection Head

Few-shot learning models based on imprinted weights have achieved excellent results on several benchmarks. In these methods, the network model directly sets the weights of the final layers for novel classes from the latent representations of the training classes. As a result, the learned representations lead to good performance accuracy in training classes. However, the performance accuracy may be poor on unseen classes. This paper provides an alternative training technique for imprinted weight models. We find that adding projection heads can yield substantial improvements over the baseline model. Our experiments show that (1) introducing nonlinear projection heads in-between the feature extractor and the classifier substantially improves generalization, (2) imprinting from the task-specific layer does not provide better generalization for novel classes. Instead, we propose imprinting from the task-agnostic layer, and (3) our design choice benefits from a large latent dimension. We validate our findings by achieving 5.6 and 4.1% improvement on the MNIST dataset trained with the Omniglot dataset