Short title: Inferring probe shape and molecular structure from AFM images

Atomic force microscopy (AFM) can visualize functional biomolecules near the physiological condition, but the observed data are limited to the surface height of specimens. Since the AFM images highly depend on the probe tip shape, for successful inference of molecular structures from the measurement, the knowledge of the probe shape is required, but is often missing. Here, we developed a method of the rigid-body fitting to AFM images, which simultaneously finds the shape of the probe tip and the placement of the molecular structure via an exhaustive search. First, we examined four similarity scores via twin-experiments for four test proteins, finding that the cosine similarity score generally worked best, whereas the pixel-RMSD and the correlation coefficient were also useful. We then applied the method to two experimental high-speed-AFM images inferring the probe shape and the molecular placement. The results suggest that the appropriate similarity score can differ between target systems. For an actin filament image, the cosine similarity apparently worked best. For an image of the flagellar protein FlhAC, we found the correlation coefficient gave better results. This difference may partly be attributed to the flexibility in the target molecule, ignored in the rigid-body fitting. The inferred tip shape and placement results can be further refined by other methods, such as the flexible fitting molecular dynamics simulations. The developed software is publicly available.