Knowledge Prompt Makes Composed Pre-Trained Models Zero-Shot News Captioner

News image captioning aims to generate descriptions containing concrete named entities for news images by leveraging relevant news articles. However, existing approaches suffer from two shortcomings: 1) lack of commonsense knowledge required to understand named entities, and 2) limited multimodal context modeling capabilities. In this paper, we propose to migrate the ability of large-scale pre-trained models for news image captioning. To acquire factual knowledge for describing named entities, we induce a pre-trained language model for commonsense knowledge reasoning using context-aware knowledge prompts. To compose a new multimodal context modeling capability, we coordinate pre-trained models by a unified language representation and constrain joint multimodal context reasoning with cross-modal consistency objective. Experimental results on GoodNews and NYTimes datasets show that our proposed method exhibits considerable captioning capabilities even without training on news data.