Deep learning is a potent tool for image segmentation, but it typically demands abundant annotated data for effective training. In scientific domains, such as cell imaging, obtaining annotations for every structure can be prohibitively expensive. Few-shot learning, adapting from one task to another using minimal examples, can alleviate the need for large training data sets. However, there is limited research addressing the particularities of using few-shot learning for cell imaging. Here, we propose a few-shot learning solution designed to be applicable to cell microscopy images. Our method trains feature extractor networks on classes with abundant labelled samples. These feature extractors are then used to generate high-resolution feature maps from the few labelled images of the new class. Finally, we train a perceptron to recombine the feature maps into predicting the new class. On two challenging cell segmentation data sets, we achieve, using five annotated images, Dice scores that are, on average, less than 20

Understanding the structural and functional mechanisms of bacteriophage 7-7-1, the flagellotropic phage infecting Agrobacterium sp. H13-3, offers promising insights into phage-host interactions. Using single particle analysis (SPA) cryo-electron microscopy (cryo-EM), we determined the capsid and tail structure, and built atomic models of capsid hexamers, pentamers and tail. Combined with cryo-electron tomography (cryo-ET) and machine learning methodologies, our findings indicate that phage 7-7-1 uses capsid fibers to establish initial contact with the host flagellum, followed by subsequent attachment to cell surface receptors. Proteinase K treatment confirmed the time-dependent degradation of capsid fibers. The study also demonstrated that capsid fibers are flexible and can interact with other phages and host flagella, suggesting a cooperative infection strategy. These results provide crucial structural insights and may open avenues for developing phage-based therapeutics against resistant bacterial pathogens. ### Competing Interest Statement The authors have declared no competing interest.

X-ray computed tomography (CT) is a powerful imaging tool for defect detection, segmentation and feature extraction in industrial applications as it enables non-destructive evaluation. The presence of artifacts and noise, however, imposes difficulties on the defect detection due to low contrast between defects and material. Different methods, the most recent ones based on deep learning, have been proposed to address both CT artifacts and noise. In this work, three state-of-the-art denoising techniques, namely BM3D, a supervised UNet and Noise2Inverse are compared with a novel approach based on a set of CNNs applied at different stages of the CT data acquisition pipeline. The comparison is done regarding the capabilities of the methods in terms of image quality enhancement as a preprocessing step in CT-based data analysis before performing defect segmentation.

Computed tomography (CT) using synchrotron radiation is a powerful technique that, compared with laboratory CT techniques, boosts high spatial and temporal resolution while also providing access to a range of contrast-formation mechanisms. The acquired projection data are typically processed by a computational pipeline composed of multiple stages. Artifacts introduced during data acquisition can propagate through the pipeline and degrade image quality in the reconstructed images. Recently, deep learning has shown significant promise in enhancing image quality for images representing scientific data. This success has driven increasing adoption of deep learning techniques in CT imaging. Various approaches have been proposed to incorporate deep learning into computational pipelines, but each has limitations in addressing artifacts effectively and efficiently in synchrotron CT, either in properly addressing the specific artifacts or in computational efficiency. Recognizing these challenges, we introduce a novel method that incorporates separate deep learning models at each stage of the tomography pipeline - projection, sinogram and reconstruction - to address specific artifacts locally in a data-driven way. Our approach includes bypass connections that feed both the outputs from previous stages and raw data to subsequent stages, minimizing the risk of error propagation. Extensive evaluations on both simulated and real-world datasets illustrate that our approach effectively reduces artifacts and outperforms comparison methods.

Computed Tomography (CT) is pivotal in industrial quality control and medical diagnostics. Sparse-view CT, offering reduced ionizing radiation, faces challenges due to its under-sampled nature, leading to ill-posed reconstruction problems. Recent advancements in Implicit Neural Representations (INRs) have shown promise in addressing sparse-view CT reconstruction. Recognizing that CT often involves scanning similar subjects, we propose a novel approach to improve reconstruction quality through joint reconstruction of multiple objects using INRs. This approach can potentially utilize the advantages of INRs and the common patterns observed across different objects. While current INR joint reconstruction techniques primarily focus on speeding up the learning process, they are not specifically tailored to enhance the final reconstruction quality. To address this gap, we introduce a novel INR-based Bayesian framework integrating latent variables to capture the common patterns across multiple objects under joint reconstruction. The common patterns then assist in the reconstruction of each object via latent variables, thereby improving the individual reconstruction. Extensive experiments demonstrate that our method achieves higher reconstruction quality with sparse views and remains robust to noise in the measurements as indicated by common numerical metrics. The obtained latent variables can also serve as network initialization for the new object and speed up the learning process.

Sparse-view Computed Tomography (CT) is favored over standard CT for its reduced ionizing radiation but poses challenges due to its inherently ill-posed nature arising from undersampled measurement data. Implicit Neural Representations (INRs) have emerged as a promising solution, demonstrating effectiveness in sparse-view CT reconstruction. Given that modern CT often scans similar subjects, we propose to improve reconstruction quality via joint reconstruction of multiple objects using INRs. This approach can potentially leverage both the strengths of INRs and the statistical regularities across multiple objects. While existing techniques of INR joint reconstruction focus on enhancing convergence rates through meta-initialization, they do not optimize for final reconstruction quality. To fill this gap, we introduce a novel INR-based Bayesian framework that incorporates latent variables to capture inter-object relationships. These latent variables act as a continuously updated reference during the optimization process, thereby enhancing the quality of individual reconstructions. We conduct extensive experiments to evaluate various aspects such as reconstruction quality, susceptibility to overfitting, and generalizability. Results demonstrate that our method sets a new standard in CT reconstruction performance. Our code will be released.

Summary Computer vision-based wood identification has been successfully applied to recognize tree species using digital images of wood sections or surfaces. However, this image-to-species approach can only recognize a limited number of species due to two main reasons: 1) the lack of a good reference database requiring high-quality standardized images from multiple individuals of hundreds or even thousands of traded timber species, and 2) species not included in the reference database cannot be identified without expert knowledge. Another bottleneck is that the feature extraction process used by these species recognition approaches is a black box, thereby creating a discrepancy between machine learning features and wood anatomical features. This discrepancy prevents wood anatomists from understanding how these machine-learning algorithms work. Here, we survey currently existing methods used in feature extraction, classification, and deep learning methods applied in wood identification along with their pitfalls and opportunities. As an example of how the field could move forward, we launch the idea of building an image-to-features-to-species identification approach based on microscopic wood images as well as text files comprising wood anatomical descriptions. If we can manage machine learning-based algorithms to recognize the main wood anatomical traits that experts use to identify species in a (semi-)automated way, this would boost wood identification in two ways: (1) extensive reference databases for each species would become less crucial as the databases are ordered at the trait level, (2) timber identification would become more feasible for species that have not yet been included in the reference database as long as wood anatomical descriptions are available.

X-ray phase contrast micro-computed tomography (micro-CT) can achieve higher contrast than conventional absorption-based X-ray micro-CT by utilizing refraction in addition to attenuation. In this work, we focus on a specific X-ray phase contrast technique, edge illumination (EI) micro-CT. EI uses a sample mask with transmitting apertures that split the X-ray beam into narrow beamlets, enabling detection of refraction-included intensity variations. Between the typical mask designs (circular and slit-shaped apertures), slit-shaped apertures offer practical advantages over circular ones, as they only require sample stepping in one direction, thereby reducing scanning time. However, this leads to anisotropic resolution, as the slit-shaped apertures enhances resolution only along the direction orthogonal to the slits. To address this limitation, we propose a self-supervised method that trains on high-resolution in-plane images to enhance resolution for out-of-plane images, effectively mitigating anisotropy. Our results on both simulated and real EI micro-CT datasets demonstrate the effectiveness of the proposed method.

Automating the detection and classification of archaeological features in remotely sensed imagery poses significant challenges due to image variability and the heterogeneity of archaeological objects, which traditional object-based solutions have struggled to fully address. However, recent advancements in deep learning offer promising avenues for overcoming these obstacles. Despite their widespread adoption, deep learning algorithms still face problems such as the lack of annotated data and the opacity of model decision-making processes. We address these challenges through an approach linking annotation scarcity and model explainability. We propose leveraging explainability techniques to not only gain insights into model decision-making processes but also to reduce annotation costs. Through a case study mapping ancient settlement mounds in Upper Mesopotamia, we apply three explainability methods to three widely-used deep learning architectures that were trained to recognize whether or not a site is present within an image. We then employ the output of the explainability methods to both delineate the sites’ boundaries, a more time-consuming process for a human expert, and to generate meaningful visual insights of the features found relevant by the networks for which we provide detailed interpretations. Additionally, we propose a new variant of an explainability method that produces site boundaries that are more consistent with the expert estimations than the existing solutions.

Many of the recent successes of deep learning-based approaches have been enabled by a framework of flexible, composable computational blocks with their parameters adjusted through an automatic differentiation mechanism to implement various data processing tasks. In this work, we explore how the same philosophy can be applied to existing “classical” (i.e., non-learning) algorithms, focusing on computed tomography (CT) as application field. We apply four key design principles of this approach for CT workflow design: end-to-end optimization, explicit quality criteria, declarative algorithm construction by building the forward model, and use of existing classical algorithms as computational blocks. Through four case studies, we demonstrate that auto-differentiation is remarkably effective beyond the boundaries of neural-network training, extending to CT workflows containing varied combinations of classical and machine learning algorithms.

Computed tomography (CT) is a widely used non-invasive medical imaging technique for disease diagnosis. The diagnostic accuracy is often affected by image resolution, which can be insufficient in practice. For medical CT images, the through-plane resolution is often worse than the in-plane resolution and there can be overlap between slices, causing difficulties in diagnoses. Self-supervised methods for through-plane resolution enhancement, which train on in-plane images and infer on through-plane images, have shown promise for both CT and MRI imaging. However, existing self-supervised methods either neglect overlap or can only handle specific cases with fixed combinations of resolution and overlap. To address these limitations, we propose a self-supervised method called SR4ZCT. It employs the same off-axis training approach while being capable of handling arbitrary combinations of resolution and overlap. Our method explicitly models the relationship between resolutions and voxel spacings of different planes to accurately simulate training images that match the original through-plane images. We highlight the significance of accurate modeling in self-supervised off-axis training and demonstrate the effectiveness of SR4ZCT using a real-world dataset.

Deep-learning algorithms for cell segmentation typically require large data sets with high-quality annotations to be trained with. However, the annotation cost for obtaining such sets may prove to be prohibitively expensive. Our work aims to reduce the time necessary to create high-quality annotations of cell images by using a relatively small well-annotated data set for training a convolutional neural network to upgrade lower-quality annotations, produced at lower annotation costs. We investigate the performance of our solution when upgrading the annotation quality for labels affected by three types of annotation error: omission, inclusion, and bias. We observe that our method can upgrade annotations affected by high error levels from 0.3 to 0.9 Dice similarity with the ground-truth annotations. We also show that a relatively small well-annotated set enlarged with samples with upgraded annotations can be used to train better-performing cell segmentation networks compared to training only on the well-annotated set. Moreover, we present a use case where our solution can be successfully employed to increase the quality of the predictions of a segmentation network trained on just 10 annotated samples.

Abstract Automated analysis of the inner ear anatomy in radiological data instead of time-consuming manual assessment is a worthwhile goal that could facilitate preoperative planning and clinical research. We propose a framework encompassing joint semantic segmentation of the inner ear and anatomical landmark detection of helicotrema, oval and round window. A fully automated pipeline with a single, dual-headed volumetric 3D U-Net was implemented, trained and evaluated using manually labeled in-house datasets from cadaveric specimen ( $$N=43$$ N = 43 ) and clinical practice ( $$N=9$$ N = 9 ). The model robustness was further evaluated on three independent open-source datasets ( $$N = 23{} + 7{} + 17$$ N = 23 + 7 + 17 scans) consisting of cadaveric specimen scans. For the in-house datasets, Dice scores of $$\text{0.97 and 0.94}$$ 0.97 and 0.94 , intersection-over-union scores of $$\text{0.94 and 0.89}$$ 0.94 and 0.89 and average Hausdorff distances of $$0.065{}$$ 0.065 and $$0.14{}$$ 0.14 voxel units were achieved. The landmark localization task was performed automatically with an average localization error of $$\text{3.3 and 5.2}$$ 3.3 and 5.2 voxel units. A robust, albeit reduced performance could be attained for the catalogue of three open-source datasets. Results of the ablation studies with 43 mono-parametric variations of the basal architecture and training protocol provided task-optimal parameters for both categories. Ablation studies against single-task variants of the basal architecture showed a clear performance benefit of coupling landmark localization with segmentation and a dataset-dependent performance impact on segmentation ability.

In x-ray computed tomography (CT), the achievable image resolution is typically limited by several pre-fixed characteristics of the x-ray source and detector. Structuring the x-ray beam using a mask with alternating opaque and transmitting septa can overcome this limit. However, the use of a mask imposes an undersampling problem: to obtain complete datasets, significant lateral sample stepping is needed in addition to the sample rotation, resulting in high x-ray doses and long acquisition times. Cycloidal CT, an alternative scanning scheme by which the sample is rotated and translated simultaneously, can provide high aperture-driven resolution without sample stepping, resulting in a lower radiation dose and faster scans. However, cycloidal sinograms are incomplete and must be restored before tomographic images can be computed. In this work, we demonstrate that high-quality images can be reconstructed by applying the recently proposed Mixed Scale Dense (MS-D) convolutional neural network (CNN) to this task. We also propose a novel training approach by which training data are acquired as part of each scan, thus removing the need for large sets of pre-existing reference data, the acquisition of which is often not practicable or possible. We present results for both simulated datasets and real-world data, showing that the combination of cycloidal CT and machine learning-based data recovery can lead to accurate high-resolution images at a limited dose.

When studying bone fragility diseases, it is difficult to identify which factors reduce bone’s resistance to fracture because these diseases alter bone at many length scales. Here, we investigate the contribution of nanoscale collagen behavior on macroscale toughness and microscale toughening mechanisms using a bovine heat-treatment fragility model. This model is assessed by developing an in situ toughness testing technique for synchrotron radiation micro-computed tomography to study the evolution of microscale crack growth in 3D. Low-dose imaging is employed with deep learning to denoise images while maintaining bone’s innate mechanical properties. We show that collagen damage significantly reduces macroscale toughness and post-yield properties. We also find that bone samples with a compromised collagen network have reduced amounts of crack deflection, the main microscale mechanism of fracture resistance. This research demonstrates that collagen damage at the nanoscale adversely affects bone’s toughening mechanisms at the microscale and reduces the overall toughness of bone.

Tomographic algorithms are often compared by evaluating them on certain benchmark datasets. For fair comparison, these datasets should ideally (i) be challenging to reconstruct, (ii) be representative of typical tomographic experiments, (iii) be flexible to allow for different acquisition modes, and (iv) include enough samples to allow for comparison of data-driven algorithms. Current approaches often satisfy only some of these requirements, but not all. For example, real-world datasets are typically challenging and representative of a category of experimental examples, but are restricted to the acquisition mode that was used in the experiment and are often limited in the number of samples. Mathematical phantoms are often flexible and can sometimes produce enough samples for data-driven approaches, but can be relatively easy to reconstruct and are often not representative of typical scanned objects. In this paper, we present a family of foam-like mathematical phantoms that aims to satisfy all four requirements simultaneously. The phantoms consist of foam-like structures with more than 100000 features, making them challenging to reconstruct and representative of common tomography samples. Because the phantoms are computer-generated, varying acquisition modes and experimental conditions can be simulated. An effectively unlimited number of random variations of the phantoms can be generated, making them suitable for data-driven approaches. We give a formal mathematical definition of the foam-like phantoms, and explain how they can be generated and used in virtual tomographic experiments in a computationally efficient way. In addition, several 4D extensions of the 3D phantoms are given, enabling comparisons of algorithms for dynamic tomography. Finally, example phantoms and tomographic datasets are given, showing that the phantoms can be effectively used to make fair and informative comparisons between tomography algorithms.

Recent studies have shown that there can be high inter- and intra-observer variability when creating annotations for biomedical image segmentation. To mitigate the effects of manual annotation variability when training machine learning algorithms, various methods have been developed. However, little work has been done on actually assessing the impact of annotation errors on machine learning-based segmentation. For the task of cell segmentation, our work aims to bridge this gap by providing a thorough analysis of three types of potential annotation errors. We tackle the limitation of previous studies that lack a golden standard ground truth by performing our analysis on two synthetically-generated data sets with perfect labels, while also validating our observations on manually-labeled data. Moreover, we discuss the influence of the annotation errors on the results of three different network architectures: UNet, SegNet, and MSD. We find that UNet shows the overall best robustness for all data sets on two categories of errors, especially when the severity of the error is low, while MSD generalizes well even when a large proportion of the cell labels is missing during training. Moreover, we observe that special care should be taken to avoid wrongly labeling large objects when the target cells have small footprints.

Host cell invasion by intracellular, eukaryotic parasites, like the many important species within the phylum Api-complexa, is a remarkable and active process involving the coordinated action of many apical organelles and other structures. To date, capturing how these various structures interact during invasion has been difficult to observe in detail. Here, we used cryogenic electron tomography to generate images of the apical complex of Toxoplasma gondii tachyzoites under conditions that mimic resting parasites and those primed to invade through addition of a calcium ionophore. Using AI-based image-processing we were able to annotate 48 tomograms to identify and extract densities of the relevant subcellular organelles and accurately analyze features in 3D. We describe an interaction between an anteriorly located apical vesicle and a rhoptry tip that occurs only in the ionophore-stimulated parasites and that is associated with dramatic changes in the vesicle’s shape in what appears to be a stalled fusion event. We also present information to support the presumption that this vesicle originates from the well-described vesicles that parallel the intraconoidal microtubules and that the latter two structures are linked by a novel tether. Lastly, we show that a previously described rosette is found associated with more than just the anterior-most apical vesicle, indicating that multiple such vesicles are primed to enable rhoptry secretion.