Automatic Weather Detection Using Vision Transformers and Adaptive Transfer Learning

The ability to recognize weather patterns significantly impacts various aspects of daily life, such as weather forecasting, transportation, agriculture, and forest management. Machine learning techniques, particularly Convolutional Neural Networks (CNNs), are expected to provide more accurate weather pattern analysis than traditional radar systems. However, CNNs often struggle to capture multi-level dependencies in input images, and increasing the convolution filter size to address this issue also raises network complexity. Recently, Transformers, initially successful in Natural Language Processing (NLP), have been applied to computer vision tasks, with Vision Transformers (ViT) outperforming CNNs in many scenarios. This paper proposes a deep learning-based method for accurate detection and categorization of weather conditions into multiple classes using a transfer learning technique. Recognizing the challenge of determining the optimal number of trainable layers within a CNN and blocks within a ViT, we introduce an Adaptive Layer Freezing (ALF) technique to dynamically adjust trainable layers for optimized performance during the transfer learning process, referred to as Adaptive Transfer Learning. We assessed a new architecture using ViT-L32 and compared it to two CNN models: EfficientNetB0 and MobileNetV2. The models' performances were evaluated on two weather imaging databases, with ViT-L32 achieving the highest classification accuracy in both binary and multi-class classifications with 98.28 % and 96.45 % respectively.