Adaptive Focal Loss for Keypoint-Based Deep Learning Detectors Addressing Class Imbalance

Keypoint-based deep learning detectors have proven highly effective in object detection tasks by predicting specific keypoints to determine object classification and location. Examples include CornerNet, CenterNet, ExtremeNet, RetinaNet, FCOS, and ObjectBox. Despite their strengths, these methods are particularly susceptible to class imbalance, which can result in poor detection performance for less frequent classes. Popular solutions such as hard sampling, soft sampling, and sampling-free methods have been proposed to tackle this issue. However, these approaches often have inherent limitations, including sensitivity to hyperparameters and neglecting the gradient dynamics of the loss function. To address these challenges, this paper proposes Adaptive Focal Loss (AFL), which combines the strengths of Focal Loss (FL) and Class-Balanced Loss (CBL) while introducing an Adaptive Gradient Function. This function is specifically designed to mitigate the impact of large gradients during the early stages of training. Extensive experiments demonstrate that AFL significantly outperforms classical sampling methods across key metrics on the MS COCO, LVIS, and PASCAL VOC datasets. Notably, on the LVIS dataset, AFL achieves an average improvement of over 3% $AP_{r}$ compared to the baseline Focal Loss across multiple keypoint-based detectors, showcasing its effectiveness in enhancing rare class accuracy. Furthermore, AFL delivers substantial improvements in the $AP_{0.5\sim 0.95}$ metric, surpassing the baseline Focal Loss by an average of over 2%, 1%, and 0.2% on the LVIS, MS COCO, and PASCAL VOC datasets, respectively. This highlights AFL’s capability to enhance the overall accuracy of foreground objects. By providing a more balanced and robust solution to class imbalance, AFL demonstrates superior performance in challenging scenarios, making it a valuable advancement in keypoint-based detection.