Seeing the Rare: Meta-Aware Pointer Networks for Long-Tailed Video Captioning

Fine-grained video captioning systems often struggle with long-tailed vocabulary distributions, where rare events and objects are significantly underrepresented. This imbalance leads to captions that are overly generic, lacking specificity and relevance for less frequent but semantically important content. To address this challenge, we propose a novel video captioning framework that integrates a hybrid pointer-generator decoder with a meta-learning strategy. The pointer-generator mechanism enables dynamic copying of rare terms from visual and semantic graphs, ensuring better alignment with underrepresented concepts. Complementarily, the meta-learning component encourages generalization to infrequent instances by simulating few-shot learning scenarios during training. Experiments on the MSVD and MSR-VTT datasets show that our method consistently outperforms strong baselines, with significant gains in rare concept recall and a 12% improvement in rare term coverage, along with notable improvements in CIDEr scores.