Reconstructing synapses from anisotropic serial electron microscopy (EM) images remains challenging, especially in restoring neural circuits. Recently, hybrid 2D/3D convolution is proposed to tackle this anisotropy problem. However, Z-axis anisotropy results in distinct structural changes and voxel-level shifts. As a result, the vanilla convolution is ineffective for recovering the instance details end-to-end from anisotropic semantic information, resulting in split/merge or deviation errors. In this study, we propose a synapse reconstruction method to rebuild spatial connections of voxels from accurate planar semantics. First, we design a region-focused Mask R-CNN with a cross-patch fusion method. Anchors and losses are constrained by instance boxes to accommodate the narrow structure of synapses. Furthermore, instance confidences are used for multi-scenario duplicate detection fusion. Second, we present a novel synaptic similarity metric on the basis of synaptic shape and spatial information. A learnable hierarchical clustering strategy based on this metric is performed to obtain the voxel connections. Experiments on anisotropic EM data show the superior performance of our method, especially on the integrity of reconstructed structures. Our code is available at https://github.com/fenglingbai/SynapseReconstruction.

Background As an extension of electron tomography (ET), serial section electron tomography (serial section ET) aims to align the tomographic images of multiple thick tissue sections together, to break through the volume limitation of the single section and preserve the sub-nanoscale voxel size. It could be applied to reconstruct the intact synapse, which expands about one micrometer and contains nanoscale vesicles. However, there are several drawbacks of the existing serial section ET methods. First, locating and imaging regions of interest (ROIs) in serial sections during the shooting process is time-consuming. Second, the alignment of ET volumes is difficult due to the missing information caused by section cutting and imaging. Here we report a workflow to simplify the acquisition of ROIs in serial sections, automatically align the volume of serial section ET, and semi-automatically reconstruct the target synaptic structure. Results We propose an intelligent workflow to reconstruct the intact synapse with sub-nanometer voxel size. Our workflow includes rapid localization of ROIs in serial sections, automatic alignment, restoration, assembly of serial ET volumes, and semi-automatic target structure segmentation. For the localization and acquisition of ROIs in serial sections, we use affine transformations to calculate their approximate position based on their relative location in orderly placed sections. For the alignment of consecutive ET volumes with significantly distinct appearances, we use multi-scale image feature matching and the elastic with belief propagation (BP-Elastic) algorithm to align them from coarse to fine. For the restoration of the missing information in ET, we first estimate the number of lost images based on the pixel changes of adjacent volumes after alignment. Then, we present a missing information generation network that is appropriate for small-sample of ET volume using pre-training interpolation network and distillation learning. And we use it to generate the missing information to achieve the whole volume reconstruction. For the reconstruction of synaptic ultrastructures, we use a 3D neural network to obtain them quickly. In summary, our workflow can quickly locate and acquire ROIs in serial sections, automatically align, restore, assemble serial sections, and obtain the complete segmentation result of the target structure with minimal manual manipulation. Multiple intact synapses in wild-type rat were reconstructed at a voxel size of 0.664 nm/voxel to demonstrate the effectiveness of our workflow. Conclusions Our workflow contributes to obtaining intact synaptic structures at the sub-nanometer scale through serial section ET, which contains rapid ROI locating, automatic alignment, volume reconstruction, and semi-automatic synapse reconstruction. We have open-sourced the relevant code in our workflow, so it is easy to apply it to other labs and obtain complete 3D ultrastructures which size is similar to intact synapses with sub-nanometer voxel size.