Transformer-Based Unsupervised Image Registration Using SSIM and Homography Loss for Steady Camera and Aerial Videos

Image registration is an essential and initial block in the pipeline of computer vision tasks and systems. It is defined as transforming a moving image into a target image with the minimum difference when aligned. Unlike previous work for general-purpose datasets, aerial images suffer from mechanical shakes, leading to deformed distortion similar to medical volumetric image registration task. Inspired by medical approaches, we use a Transformer-based network with a semi-unlimited receptive field Swin block to produce a general output for each pixel named flow matrix. Flow matrix is utilized instead of regressing parameters of transformation matrix with a fixed degree of freedom that cannot handle the structural difference between images. This leads to introducing a new loss function based on the Structural Similarity Index Measure (SSIM) and embedding Homography transformation as a regularization term. Combining a generalized-designed network and loss function based on problem definition significantly enhanced results.