I2M: a Box-Based Detection Framework for Molecular Structure Recognition from Images

Accurate recognition and reconstruction of chemical structures from molecular images is critical in cheminformatics, with applications spanning drug discovery, materials science, and patent analysis. However, much of this chemical matter exists in implicit forms that are not directly machine-readable, which limits its integration into structured databases. Traditional rule-based and machine learning-based methods often perform poorly when images contain overlapping atoms, ambiguous bonds, or low visual quality. To address these challenges, we introduce I2M, a framework integrates Vision Transformer (ViT) encoders enhanced with adapter modules and utilizes pre-trained Detection Transformer (DETR) weights for improved accuracy and efficiency. I2M identifies atoms, charges, and bonds by predicting their locations and categories, then assembles these elements into chemically interpretable molecular graphs. The model employs robust query selection mechanisms proposed in DETR, improving decoding accuracy, and advanced validation routines to effectively handle complex cases. Compared with existing models, including MolVec, DECIMER-Image-to-SMILES, DECIMER Image Transformer, MolGrapher and MolScribe, I2M achieves good performance in both accuracy and computational efficiency. Additionally, to facilitate the automatic and convenient testing of other chemical structure recognition methods in future studies, we have developed a chemical structure image benchmark and published the datasets. This work highlights I2M’s transformative potential in cheminformatics, offering a robust solution to bridge the gap between molecular imagery and digital molecular representations. I2M introduces the first box-based detection framework for molecular structure recognition, enabling direct prediction of atoms, bonds, and charges with spatial localization while preserving molecular 2D layouts. Compared with existing image-to-SMILES and graph-based recognition approaches, I2M provides an interpretable detection-based paradigm with improved accuracy and computational efficiency across diverse chemical image benchmarks. Furthermore, we establish publicly available chemical structure image benchmarks to facilitate reproducible evaluation and future development of molecular image recognition methods.