Comparative Analysis of Convolutional Neural Networks and Vision Transformer on Classification of Images Containing Homogenous Microstructures

The purpose of this work is to study the behavior of fundamentally different architectures of neural networks, and to find out whether Transformers can outperform deep Convolutional networks in analyzing specific data. The specificity of the visual data used in our experiments comes from the fact that both micro- and macrostructures of objects in images are important for classification, and micro-structure is homogenous across the image. An example of these type of data is microscopical medical images of bacterial colonies or cells. The two most popular architectures ResNet-18 and ResNet-50 were chosen to represent deep Convolutional networks, and ViT was used to represent transformers. We analyzed how patching operation, that potentially destroys macrostructure, affects recognition. As a result of the experiments, we can conclude that for such specific data, the chosen deep Convolutional networks outperform the Visual Transformer.