YOLO-MousePose: A Novel Framework and Dataset for Mouse Pose Estimation From a Top–Down View

The top-down view is particularly advantageous for mouse pose estimation as it provides a clear, unobstructed perspective of the body of the mouse, enabling more accurate behavioral analysis. However, physiological differences make existing human pose estimation algorithms less applicable to mice, and the lack of open-source datasets hinders algorithmic progress in this field. To address these challenges, we present YOLO-MousePose, an enhanced version of the YOLO-Pose specifically designed for mouse pose estimation from a top-down view. We also construct the mouse pose dataset from open-field test (MPD-OFT), which includes 14586 annotated images, with each image containing a single mouse. YOLO-MousePose uses a regression-based approach for precise keypoint localization, simultaneously detecting the bounding box and 2-D pose of the mouse in a single forward pass. The model features a fusion channel-specialized encoder (FCSE) module that selectively fuses multiscale feature information, improving sensitivity to small objects and effectively addressing the challenges of small object detection and keypoint localization. Additionally, we optimize the keypoint localization loss function to accelerate convergence and prevent performance degradation. On our open-source dataset, YOLO-MousePose achieves a root-mean-square error (RMSE) of 8.41 mm, a mean absolute error (MAE) of 5.52 mm, and an average percentage of correct keypoints (PCKs) score of 96.1% at a 0.1 threshold, matching the accuracy of the previous state-of-the-art (SOTA) algorithm GM-SCENet while using only 28% of its parameters. The dataset and code are publicly available at https://github.com/bujihao/YOLO-MousePose.