The meniscus effect is caused by surface tension of cell culture media with the walls of multi-well plates resulting in a curvature of media at the edges of a well. This introduces significant distortions in label-free optical images generated by phase contrast microscopy (PCM) images, hindering automated analysis in downstream tasks. We present a deep learningbased approach for effectively removing meniscus-induced distortions in phase contrast images acquired from well plates seeded with HEK293T cells. To train and evaluate our model, we introduce a novel imaging approach to obtain highfidelity ground truth. First, cells are fixed with paraformaldehyde and imaged using PCM with the meniscus effect present. Afterwards, a glass slide is placed atop each well and filled with phosphate buffered saline to touch the slide, resulting in flattening the buffer surface and eliminating the meniscus effect. These paired images, with and without meniscus artifacts, serve as training and testing data for our model. The resulting label-free approach contrasts with fluorescence-based labeling techniques that require staining and may negatively impact cell behavior and viability. Our model architecture is based on the Fully Convolutional Dense network. We evaluate the model on images from a separate well in regions of significant and minimal meniscus distortions. Quantitative results show a substantial correction of meniscus artifacts across both regions: the structural similarity index (SSIM) between the reconstructed images and ground truth is 0.93 on average, compared with an average SSIM of 0.56 between ground truth and the original meniscusaffected images. Visual inspection confirms effective reconstruction, preserving cellular features even in challenging edge regions of high meniscus. With the meniscus present, only 37.9% of the well area is observable, whereas model reconstruction increases the observable area to 91.3%, indicating a substantial recovery of image content. The model also demonstrates robustness to small out-of-focus imaging artifacts that blur the underlying cell content. Our results highlight the feasibility of AI-assisted correction of meniscus artifacts in PCM, significantly enhancing the use of standard culture plates for quantitative imaging studies.

Repeatability of measurements from image analytics is difficult, due to the heterogeneity and complexity of cell samples, exact microscope stage positioning, and slide thickness. We present a method to define and use a reference focal plane that provides repeatable measurements with very high accuracy, by relying on control beads as reference material and a convolutional neural network focused on the control bead images. Previously we defined a reference effective focal plane (REFP) based on the image gradient of bead edges and three specific bead image features. This paper both generalizes and improves on this previous work. First, we refine the definition of the REFP by fitting a cubic spline to describe the relationship between the distance from a bead's center and pixel intensity and by sharing information across experiments, exposures, and fields of view. Second, we remove our reliance on image features that behave differently from one instrument to another. Instead, we apply a convolutional regression neural network (ResNet 18) trained on cropped bead images that is generalizable to multiple microscopes. Our ResNet 18 network predicts the location of the REFP with only a single inferenced image acquisition that can be taken across a wide range of focal planes and exposure times. We illustrate the different strategies and hyperparameter optimization of the ResNet 18 to achieve a high prediction accuracy with an uncertainty for every image tested coming within the microscope repeatability measure of 7.5 mu m from the desired focal plane. We demonstrate the generalizability of this methodology by applying it to two different optical systems and show that this level of accuracy can be achieved using only 6 beads per image.

To facilitate the characterization of unlabeled induced pluripotent stem cells (iPSCs) during culture and expansion, we developed an AI pipeline for nuclear segmentation and mitosis detection from phase contrast images of individual cells within iPSC colonies. The analysis uses a 2D convolutional neural network (U-Net) plus a 3D U-Net applied on time lapse images to detect and segment nuclei, mitotic events, and daughter nuclei to enable tracking of large numbers of individual cells over long times in culture. The analysis uses fluorescence data to train models for segmenting nuclei in phase contrast images. The use of classical image processing routines to segment fluorescent nuclei precludes the need for manual annotation. We optimize and evaluate the accuracy of automated annotation to assure the reliability of the training. The model is generalizable in that it performs well on different datasets with an average F1 score of 0.94, on cells at different densities, and on cells from different pluripotent cell lines. The method allows us to assess, in a non-invasive manner, rates of mitosis and cell division which serve as indicators of cell state and cell health. We assess these parameters in up to hundreds of thousands of cells in culture for more than 36 hours, at different locations in the colonies, and as a function of excitation light exposure.

A method is presented to measure machine tool thermal drift for error compensation. A wireless microscope within a tool holder in the spindle is used to capture videos of image targets attached to the worktable. For each target, one video is captured during spindle rotation orthogonal to the worktable and another video is captured during axis translation orthogonal to the worktable. Data are collected periodically so that the three-dimensional thermal error at each target location is determined via image analysis. Experiments verify that the method measures micrometer-level tool-to-workpiece thermal drift for error model development and thermal drift compensation.

In the field of tissue engineering, 3D scaffolds and cells are often combined to yield constructs that are used as therapeutics to repair or restore tissue function in patients. Viable cells are often required to achieve the intended mechanism of action for the therapy, where the live cells may build new tissue or may release factors that induce tissue regeneration. Thus, there is a need to reliably measure cell viability in 3D scaffolds as a quality attribute of a tissue-engineered medical product. Here, we developed a noninvasive, label-free, 3D optical coherence tomography (OCT) method to rapidly (2.5 min) image large sample volumes (1 mm3 ) to assess cell viability and distribution within scaffolds. OCT imaging was assessed using a model scaffold-cell system consisting of a polysaccharide-based hydrogel seeded with human Jurkat cells. Four test systems were used: hydrogel seeded with live cells, hydrogel seeded with heat-shocked or fixed dead cells and hydrogel without any cells. Time series OCT images demonstrated changes in the time-dependent speckle patterns due to refractive index (RI) variations within live cells that were not observed for pure hydrogel samples or hydrogels with dead cells. The changes in speckle patterns were used to generate live-cell contrast by image subtraction. In this way, objects with large changes in RI were binned as live cells. Using this approach, on average, OCT imaging measurements counted 326 ± 52 live cells per 0.288 mm3 for hydrogels that were seeded with 288 live cells (as determined by the acridine orange-propidium iodide cell counting method prior to seeding cells in gels). Considering the substantial uncertainties in fabricating the scaffold-cell constructs, such as the error from pipetting and counting cells, a 13% difference in the live-cell count is reasonable. Additionally, the 3D distribution of live cells was mapped within a hydrogel scaffold to assess the uniformity of their distribution across the volume. Our results demonstrate a real-time, noninvasive method to rapidly assess the spatial distribution of live cells within a 3D scaffold that could be useful for assessing tissue-engineered medical products.

Collective cell migration is an umbrella term for a rich variety of cell behaviors, whose distinct character is important for biological function, notably for cancer metastasis. One essential feature of collective behavior is the motion of cells relative to their immediate neighbors. We introduce an AI-based pipeline to segment and track cell nuclei from phase-contrast images. Nuclei segmentation is based on a U-Net convolutional neural network trained on images with nucleus staining. Tracking, based on the Crocker-Grier algorithm, quantifies nuclei movement and allows for robust downstream analysis of collective motion. Because the AI algorithm required no new training data, our approach promises to be applicable to and yield new insights for vast libraries of existing collective motion images. In a systematic analysis of a cell line panel with oncogenic mutations, we find that the collective rearrangement metric, D2 min, which reflects non-affine motion, shows promise as an indicator of metastatic potential.

In order to predict cell population behavior, it is important to understand the dynamic characteristics of individual cells. Individual induced pluripotent stem (iPS) cells in colonies have been difficult to track over long times, both because segmentation is challenging due to close proximity of cells and because cell morphology at the time of cell division does not change dramatically in phase contrast images; image features do not provide sufficient discrimination for 2D neural network models of label-free images. However, these cells do not move significantly during division, and they display a distinct temporal pattern of morphologies. As a result, we can detect cell division with images overlaid in time. Using a combination of a 3D neural network applied over time-lapse data to find regions of cell division activity, followed by a 2D neural network for images in these selected regions to find individual dividing cells, we developed a robust detector of iPS cell division. We created an initial 3D neural network to find 3D image regions in (x,y,t) in which identified cell divisions occurred, then used semi-supervised training with additional stacks of images to create a more refined 3D model. These regions were then inferenced with our 2D neural network to find the location and time immediately before cells divide when they contain two sets of chromatin, information needed to track the cells after division. False positives from the 3D inferenced results were identified and removed with the addition of the 2D model. We successfully identified 37 of the 38 cell division events in our manually annotated test image stack, and specified the time and (x,y) location of each cell just before division within an accuracy of 10 pixels.

The properties and structure of the cellular microenvironment can influence cell behavior. Sites of cell adhesion to the extracellular matrix (ECM) initiate intracellular signaling that directs cell functions such as proliferation, differentiation, and apoptosis. Electrospun fibers mimic the fibrous nature of native ECM proteins and cell culture in fibers affects cell shape and dimensionality, which can drive specific functions, such as the osteogenic differentiation of primary human bone marrow stromal cells (hBMSCs), by. In order to probe how scaffolds affect cell shape and behavior, cell-fiber contacts were imaged to assess their shape and dimensionality through a novel approach. Fluorescent polymeric fiber scaffolds were made so that they could be imaged by confocal fluorescence microscopy. Fluorescent polymer films were made as a planar control. hBSMCs were cultured on the fluorescent substrates and the cells and substrates were imaged. Two different image analysis approaches, one having geometrical assumptions and the other having statistical assumptions, were used to analyze the 3D structure of cell-scaffold contacts. The cells cultured in scaffolds contacted the fibers in multiple planes over the surface of the cell, while the cells cultured on films had contacts confined to the bottom surface of the cell. Shape metric analysis indicated that cell-fiber contacts had greater dimensionality and greater 3D character than the cell-film contacts. These results suggest that cell adhesion site-initiated signaling could emanate from multiple planes over the cell surface during culture in fibers, as opposed to emanating only from the cell's basal surface during culture on planar surfaces.

This chapter addresses object measurements from 2D microscopy images. Our motivation is to introduce readers to image-based object measurements and to quantify numerical variability of image features and feature-based classification outcomes. The objective is to highlight the sources of digital image processing variability when deriving image-based scientific conclusions. By characterizing feature variations across Python scikit-image, CellProfiler, MaZda, ImageJ, and in-house Java libraries, we concluded 15.6% of 32 intensity features, 47.9% of 71 shape features, and 88.2% of 68 textural features differ in values. Shape and textural feature variations had a negative impact on classification outcomes in 52.9% and 97.1% of all single feature-based classifications, respectively. All these numerical results are available at https://isg.nist.gov/deepzoomweb/resources/featureVariability/index.html and are traceable to image objects. They document the amount of digital image processing noise and the importance of gathering computational provenance information.

Trypan blue dye exclusion-based cell viability measurements are highly dependent upon image quality and consistency. In order to make measurements repeatable, one must be able to reliably capture images at a consistent focal plane, and with signal-to-noise ratio within appropriate limits to support proper execution of image analysis routines. Imaging chambers and imaging systems used for trypan blue analysis can be inconsistent or can drift over time, leading to a need to assure the acquisition of images prior to automated image analysis. Although cell-based autofocus techniques can be applied, the heterogeneity and complexity of the cell samples can make it difficult to assure the effectiveness, repeatability and accuracy of the routine for each measurement. Instead of auto-focusing on cells in our images, we add control beads to the images, and use them to repeatedly return to a reference focal plane. We use bead image features that have stable profiles across a wide range of focal values and exposure levels. We created a predictive model based on image quality features computed over reference datasets. Because the beads have little variation, we can determine the reference plane from bead image features computed over a single-shot image and can reproducibly return to that reference plane with each sample. The achieved accuracy (over 95%) is within the limits of the actuator repeatability. We demonstrate that a small number of beads (less than 3 beads per image) is needed to achieve this accuracy. We have also developed an open-source Graphical User Interface called Bead Benchmarking-Focus And Intensity Tool (BB-FAIT) to implement these methods for a semi-automated cell viability analyser.

A modern day light microscope has evolved from a tool devoted to making primarily empirical observations to what is now a sophisticated , quantitative device that is an integral part of both physical and life science research. Nowadays, microscopes are found in nearly every experimental laboratory. However, despite their prevalent use in capturing and quantifying scientific phenomena, neither a thorough understanding of the principles underlying quantitative imaging techniques nor appropriate knowledge of how to calibrate, operate and maintain microscopes can be taken for granted. This is clearly demonstrated by the well-documented and widespread difficulties that are routinely encountered in evaluating acquired data and reproducing scientific experiments. Indeed, studies have shown that more than 70% of researchers have tried and failed to repeat another scientist's experiments, while more than half have even failed to reproduce their own experiments. One factor behind the reproducibility crisis of experiments published in scientific journals is the frequent underreporting of imaging methods caused by a lack of awareness and/or a lack of knowledge of the applied technique. Whereas quality control procedures for some methods used in biomedical research, such as genomics (e.g. DNA sequencing, RNA-seq) or cytometry, have been introduced (e.g. ENCODE), this issue has not been tackled for optical microscopy instrumentation and images. Although many calibration standards and protocols have been published, there is a lack of awareness and agreement on common standards and guidelines for quality assessment and reproducibility. In April 2020, the QUality Assessment and REProducibility for instruments and images in Light Microscopy (QUAREP-LiMi) initiative was formed. This initiative comprises imaging scientists from academia and industry who share a common interest in achieving a better understanding of the performance and limitations of microscopes and improved quality control (QC) in light microscopy. The ultimate goal of the QUAREP-LiMi initiative is to establish a set of common QC standards, guidelines, metadata models and tools, including detailed protocols, with the ultimate aim of improving reproducible advances in scientific research. This White Paper (1) summarizes the major obstacles identified in the field that motivated the launch of the QUAREP-LiMi initiative; (2) identifies the urgent need to address these obstacles in a grassroots manner, through a community of stakeholders including, researchers, imaging scientists, bioimage analysts, bioimage informatics developers, corporate partners, funding agencies, standards organizations, scientific publishers and observers of such; (3) outlines the current actions of the QUAREP-LiMi initiative and (4) proposes future steps that can be taken to improve the dissemination and acceptance of the proposed guidelines to manage QC. To summarize, the principal goal of the QUAREP-LiMi initiative is to improve the overall quality and reproducibility of light microscope image data by introducing broadly accepted standard practices and accurately captured image data metrics.

We quantify the robustness of the semantic segmentation model U-Net, applied to single cell nuclei detection, with respect to the following factors: (1) automated vs manual training annotations, (2) quantity of training data, and (3) microscope image focus. The difficulty of obtaining sufficient volumes of accurate manually annotated training data to create an accurate Convolutional Neural Networks (CNN) model is overcome by the temporary use of fluorescent labels to automate the creation of training datasets using traditional image processing algorithms. The accuracy measurement is computed with respect to manually annotated masks which were also created to evaluate the effectiveness of using automated training set generation via the fluorescent images. The metric to compute the accuracy is the false positive/negative rate of cell nuclei detection. The goal is to maximize the true positive rate while minimizing the false positive rate. We found that automated segmentation of fluorescently labeled nuclei provides viable training data without the need for manual segmentation. A training dataset size of four large stitched images with medium cell density was enough to reach a true positive rate above 88% and a false positive rate below 20%.

Increases in the number of cell therapies in the preclinical and clinical phases have prompted the need for reliable and non-invasive assays to validate transplant function in clinical biomanufacturing. We developed a robust characterization methodology composed of quantitative bright-field absorbance microscopy (QBAM) and deep neural networks (DNNs) to non-invasively predict tissue function and cellular donor identity. The methodology was validated using clinical-grade induced pluripotent stem cell derived retinal pigment epithelial cells (iPSC-RPE). QBAM images of iPSC-RPE were used to train DNNs that predicted iPSC-RPE monolayer transepithelial resistance, predicted polarized vascular endothelial growth factor (VEGF) secretion, and matched iPSC-RPE monolayers to the stem cell donors. DNN predictions were supplemented with traditional machine learning algorithms that identified shape and texture features of single cells that were used to predict tissue function and iPSC donor identity. These results demonstrate non-invasive cell therapy characterization can be achieved with QBAM and machine learning.

Induced pluripotent stem cells (iPSCs) are reprogrammed cells that can have heterogeneous biological potential. Quality assurance metrics of reprogrammed iPSCs will be critical to ensure reliable use in cell therapies and personalized diagnostic tests. We present a quantitative phase imaging (QPI) workflow which includes acquisition, processing, and stitching multiple adjacent image tiles across a large field of view (LFOV) of a culture vessel. Low magnification image tiles (10x) were acquired with a Phasics SID4BIO camera on a Zeiss microscope. iPSC cultures were maintained using a custom stage incubator on an automated stage. We implement an image acquisition strategy that compensates for non-flat illumination wavefronts to enable imaging of an entire well plate, including the meniscus region normally obscured in Zernike phase contrast imaging. Polynomial fitting and background mode correction was implemented to enable comparability and stitching between multiple tiles. LFOV imaging of reference materials indicated that image acquisition and processing strategies did not affect quantitative phase measurements across the LFOV. Analysis of iPSC colony images demonstrated mass doubling time was significantly different than area doubling time. These measurements were benchmarked with prototype microsphere beads and etched-glass gratings with specified spatial dimensions designed to be QPI reference materials with optical pathlength shifts suitable for cell microscopy. This QPI workflow and the use of reference materials can provide non-destructive traceable imaging method for novel iPSC heterogeneity characterization.