Calibrating the Principal Point of Vehicle-Mounted Fisheye Cameras Using Point-Oriented Representation

Estimating the principal point for vehicle-mounted fisheye cameras is essential for accurate vision tasks. However, traditional calibration methods require human intervention and specific patterns, lacking flexibility and adaptability to different camera models during camera change. In this study, we propose an automatic, flexible, and end-to-end self-calibration framework that predicts the principal point from a single fisheye image. First, considering the dimensional differences between the high-dimensional image features and the 2-D principal point coordinates, directly regressing the parameters is unsuitable for our task. To address this, we propose a point-oriented representation map to represent the principal point locations, leveraging the distortion characteristics of the fisheye camera. This approach aligns more effectively with the dense prediction capabilities of neural networks. To enhance the accuracy of the generated principal point representation map, we introduce a contourlet-semantic extract network (CSEN), which is specially designed to extract the fisheye image’s semantic and edge features. Using these features, we create an initial principal point map, refined with edge-based offsets and resampled for precision. This process captures finer edge details in autonomous driving scenarios (e.g., lane markings), enabling the network to more accurately understand the distortion distribution within the image. For training and evaluation, we have compiled a fisheye image dataset from driving scenes featuring 20 distinct principal point locations. In addition, we introduce a circular cropping data augmentation technique that generates fisheye images with different principal points. Extensive experiments demonstrate the effectiveness and superiority of the proposed method. The code and dataset will be released at: https://github.com/BJTU-zjc/FCPPE-Net