Robust Table Detection and Structure Recognition from Heterogeneous Document Images

•We propose a new table detection and structure recognition approach named RobusTabNet to extract tables from heterogeneous document images.•For table detection, we use CornerNet as a new region proposal network for Faster R-CNN to improve localization accuracy.•For table structure recognition, we propose a new split-and-merge based approach, which contains a spatial CNN based separation line prediction module and a Grid CNN based cell merging module.•Our approach is robust to tables with complex structures, large blank spaces, as well as distorted or even curved shapes.•Our approach achieves state-of-the-art performance on both table detection and structure recognition public benchmarks.