Accurate Uncalibrated Rectification Using Feature Matching Re-Selection

Rectification is a core programming process that involves multiple views. In this study, we focus on uncalibrated cases that neglect intrinsic and extrinsic camera information. Existing uncalibrated rectification methods use feature-matching techniques to form relationships between different views and use those features to estimate optimized homography matrices. However, outliers are inevitable in feature matching. Using these outliers in a rectification process produces vertical disparity errors and unwanted geometric distortion. To tackle the problem, we propose a novel method that can learn from the rectification results, re-select the matching pairs, and find superior solutions. The proposed method introduces a novel workflow for uncalibrated rectification that incorporates three cores: field of view (FoV) neutralization, rectification, and feature matching re-selection (FMR). While the FoV neutralization module handles FoV differences among views, a combination of the rectification module and FMR module results in the optimal homography matrices. The rectification module takes neutral correspondences and estimates the optimized rectified matrices. Applying the results from the rectification module, the FMR module optimizes the correspondences and return them to the rectification module. For the rectification module, we apply adaptive geometric constraints and our updated optimization strategy to secure satisfied vertical disparity errors while maintaining low distortion levels. Multidisciplinary experiments are performed to analyze the capabilities of the proposed method. We combine existing datasets with extra samples, including various outdoor environments, to gauge the performance better. Compared with existing methods, the proposed method produces fewer rectification errors and keeps the rectified images under low distortion rates.