Shift- ViT: Siamese Vision Transformer using Shifted Branches

In this paper, we propose a novel classification method to overcome the limitation of Vision Transformer (ViT). Inspired by Transformers that performed state-of-the-art in NLP (Natural Language Process), research on ViT that applied a learning mechanism to vision tasks is being actively conducted. However, ViT has no inductive bias fundamentally unlike conventional Convolution Neural Network (CNN) models. For that reason, previous ViTs are limited to handle sensitively to small changes in the image. Therefore, in this paper, we propose Shift-ViT, a new algorithm that adaptively learns image changes by extending the conventional ViT. Our proposed model attempts to overcome the limitations of the existing ViT by using Siamese encoders to learn both horizontal and vertical features of the raw input image.