DeformableFormer for Classification of the Presence or Absence of Pancreatic Tissue Fragments in Pancreatic Disease

The purpose of this paper is to classify from an unstained image whether it is available for examination or not, and to exceed the accuracy of visual classification by specialist physicians by machine learning. Currently, Vision Transformer(ViT) and MetaFormer based PoolFormer have shown high accuracy in the image classification task. However, the pancreatic tissue fragment is a part of the image and has a complex shape, so the Vision Transformer, which processes the entire image, and the PoolFormer, which uses localized but fixed Convolution and Pooling, cannot classify it well. To address the problem, we require localized and image-specific feature extraction depending on the shape of a target. Therefore, we propose DeformableFormer, which enables local and dynamic feature extraction depending on the shape of the classification target in each image. To evaluate our method, we classify two categories of pancreatic tissue fragments; available and unavailable for examination. We demonstrated that our method outperformed the accuracy by specialist physicians and conventional methods such as ViT, Poolformer and the method using contrastive learning.