Quantifying noise limitations of neural network segmentations in high-resolution transmission electron microscopy

Motivated by the need for low electron dose transmission electron microscopy imaging, we report the optimal frame dose (i.e. e−/Å2) range for object detection and segmentation tasks with neural networks. The MSD-net architecture shows promising abilities over the industry standard U-net architecture in generalising to frame doses below the range included in the training set, for both simulated and experimental images. It also presents a heightened ability to learn from lower dose images. The MSD-net displays mild visibility of a Au nanoparticle at 20–30 e−/Å2, and converges at 200 e−/Å2 where a full segmentation of the nanoparticle is achieved. Between 30 and 200 e−/Å2 object detection applications are still possible. This work also highlights the importance of modelling the modulation transfer function when training with simulated images for applications on images acquired with scintillator based detectors such as the Gatan Oneview camera. A parametric form of the modulation transfer function is applied with varying ranges of parameters, and the effects on low electron dose segmentation is presented.