A Formative Study on Designing Accurate and Natural Figure Captioning Systems

Automatic figure captioning is widely useful for improving the readability and accessibility of figures. Despite recent advances in figure question answering and parsing figure elements that enable machines to accurately read information from figures, the machine learning community still lacks sufficient understanding of this problem, on what contents are important to include in a caption and how to make it sound natural. In this work, we crawled, annotated, and analyzed a corpus of real-world human-written figure captions. Our study results show that real-world captions usually consist of a finite set of caption units and that automatic figure captioning should be formulated as a multi-stage task. The first stage is to generate caption units with high accuracy and the second is to stitch together the units with diverse stitching patterns, to form a natural caption.