Rule-driven News Captioning
CoRR(2024)
摘要
News captioning task aims to generate sentences by describing named entities
or concrete events for an image with its news article. Existing methods have
achieved remarkable results by relying on the large-scale pre-trained models,
which primarily focus on the correlations between the input news content and
the output predictions. However, the news captioning requires adhering to some
fundamental rules of news reporting, such as accurately describing the
individuals and actions associated with the event. In this paper, we propose
the rule-driven news captioning method, which can generate image descriptions
following designated rule signal. Specifically, we first design the news-aware
semantic rule for the descriptions. This rule incorporates the primary action
depicted in the image (e.g., "performing") and the roles played by named
entities involved in the action (e.g., "Agent" and "Place"). Second, we inject
this semantic rule into the large-scale pre-trained model, BART, with the
prefix-tuning strategy, where multiple encoder layers are embedded with
news-aware semantic rule. Finally, we can effectively guide BART to generate
news sentences that comply with the designated rule. Extensive experiments on
two widely used datasets (i.e., GoodNews and NYTimes800k) demonstrate the
effectiveness of our method.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要