Understanding biological systems requires observing features and processes across vast spatial and temporal scales, spanning nanometers to centimeters and milliseconds to days, often using multiple imaging modalities within complex native microenvironments. Yet, achieving this comprehensive view is challenging because microscopes optimized for specific tasks typically lack versatility due to inherent optical and sample handling tradeoffs, and frequently suffer performance degradation from sample-induced optical aberrations in multicellular contexts. Here, we present Multimodal Optical Scope with Adaptive Imaging Correction (MOSAIC), a reconfigurable microscope that integrates multiple advanced imaging techniques including light-sheet, label-free, super-resolution and multiphoton, all equipped with adaptive optics. MOSAIC enables noninvasive imaging of subcellular dynamics in both cultured cells and live multicellular organisms, nanoscale mapping of molecular architectures across millimeter-scale expanded tissues and structural/functional neural imaging within live mice. MOSAIC facilitates correlative studies across biological scales within the same specimen, providing an integrated platform for broad biological investigation.

Optical nanoscopy of intact biological specimens has been transformed by recent advancements in hydrogel-based tissue clearing and expansion, enabling the imaging of cellular and subcellular structures with molecular contrast. However, existing high-resolution fluorescence microscopes are physically limited by objective-to-specimen distance, which prevents the study of whole-mount specimens without physical sectioning. To address this challenge, we developed a photochemical strategy for spatially precise sectioning of specimens. By combining serial photochemical sectioning with lattice light-sheet imaging and petabyte-scale computation, we imaged and reconstructed axons and myelin sheaths across entire mouse olfactory bulbs at nanoscale resolution. An olfactory bulb-wide analysis of myelinated and unmyelinated axons revealed distinctive patterns of axon degeneration and de-/dysmyelination in the neurodegenerative brain, highlighting the potential for peta- to exabyte-scale super-resolution studies using this approach.

Optical nanoscopy of intact biological specimens has been transformed by recent advancements in hydrogel-based tissue clearing and expansion, enabling the imaging of cellular and subcellular structures with molecular contrast. However, existing high-resolution fluorescence microscopes have limited imaging depth, which prevents the study of whole-mount specimens without physical sectioning. To address this challenge, we developed "photochemical sectioning," a spatially precise, light-based sample sectioning process. By combining photochemical sectioning with volumetric lattice light-sheet imaging and petabyte-scale computation, we imaged and reconstructed axons and myelination sheaths across entire mouse olfactory bulbs at nanoscale resolution. An olfactory-bulb-wide analysis of myelinated and unmyelinated axons revealed distinctive patterns of axon degeneration and de-/dysmyelination in the neurodegenerative mouse, highlighting the potential for peta- to exabyte-scale super-resolution studies using this approach.

Light sheet microscopy is a powerful technique for high-speed three-dimensional imaging of subcellular dynamics and large biological specimens. However, it often generates datasets ranging from hundreds of gigabytes to petabytes in size for a single experiment. Conventional computational tools process such images far slower than the time to acquire them and often fail outright due to memory limitations. To address these challenges, we present PetaKit5D, a scalable software solution for efficient petabyte-scale light sheet image processing. This software incorporates a suite of commonly used processing tools that are optimized for memory and performance. Notable advancements include rapid image readers and writers, fast and memory-efficient geometric transformations, high-performance Richardson-Lucy deconvolution and scalable Zarr-based stitching. These features outperform state-of-the-art methods by over one order of magnitude, enabling the processing of petabyte-scale image data at the full teravoxel rates of modern imaging cameras. The software opens new avenues for biological discoveries through large-scale imaging experiments. PetaKit5D offers versatile processing workflows for light sheet microscopy data including performant image input/output, geometric transformations, deconvolution and stitching. The software is efficient and scalable to petabyte-size datasets.

Lattice light sheet microscopy excels at the noninvasive imaging of three-dimensional (3D) dynamic processes at high spatiotemporal resolution within cells and developing embryos. Recently, several papers have called into question the performance of lattice light sheets relative to the Gaussian sheets most common in light sheet microscopy. Here, we undertake a theoretical and experimental analysis of various forms of light sheet microscopy, which demonstrates and explains why lattice light sheets provide substantial improvements in resolution and photobleaching reduction. The analysis provides a procedure to select the correct light sheet for a desired experiment and specifies the processing that maximizes the use of all fluorescence generated within the light sheet excitation envelope for optimal resolution while minimizing image artifacts and photodamage. We also introduce a new type of “harmonic balanced” lattice light sheet that improves performance at all spatial frequencies within its 3D resolution limits and maintains this performance over lengthened propagation distances allowing for expanded fields of view.

T cells can express multiple inhibitory receptors. Upon induction of T cell exhaustion in response to a persistent antigen, prominently in the anti-tumor immune response, many are expressed simultaneously. Key inhibitory receptors are CTLA-4, PD-1, LAG3, TIM3, and TIGIT, as investigated here. These receptors are important as central therapeutic targets in cancer immunotherapy. Inhibitory receptors are not constitutively expressed on the cell surface, but substantial fractions reside in intracellular vesicular structures. It remains unresolved to which extent the subcellular localization of different inhibitory receptors is distinct. Using quantitative imaging of subcellular distributions and plasma membrane insertion as complemented by proximity proteomics and biochemical analysis of the association of the inhibitory receptors with trafficking adaptors, the subcellular distributions of the five inhibitory receptors were discrete. The distribution of CTLA-4 was most distinct, with preferential association with lysosomal-derived vesicles and the sorting nexin 1/2/5/6 transport machinery. With a lack of evidence for the existence of specific vesicle subtypes to explain divergent inhibitory receptor distributions, we suggest that such distributions are driven by divergent trafficking through an overlapping joint set of vesicular structures. This extensive characterization of the subcellular localization of five inhibitory receptors in relation to each other lays the foundation for the molecular investigation of their trafficking and its therapeutic exploitation.

CD8+ T cell killing of tumor cells is suppressed by the tumor microenvironment. Inhibitory receptors, prominently PD-1, are key mediators of this suppression. To discover cellular defects triggered by tumor exposure and associated PD-1 signaling, we have established an ex vivo imaging approach to investigate CD8+ tumor infiltrating lymphocytes (TILs) interacting with tumor targets. Whilst TIL:tumor cell couples formed effectively, couple stability deteriorated within 1-2 minutes. This was associated with excessive cofilin recruitment to the cellular interface, coincident deterioration of f-actin structures, increased TIL locomotion, and impaired tumor cell killing. Diminished engagement of PD-1 within the tumor, but not acute ex vivo blockade, partially restored cell couple maintenance and killing. PD-1 thus suppresses TIL function by inducing a polarization-impaired state.

Motivation: Cell shape provides both geometry for, and a reflection of, cell function. Numerous methods for describing and modeling cell shape have been described, but previous evaluation of these methods in terms of the accuracy of generative models has been limited. Results: Here we compare traditional methods and deep autoencoders to build generative models for cell shapes in terms of the accuracy with which shapes can be reconstructed from models. We evaluated the methods on different collections of 2D and 3D cell images, and found that none of the methods gave accurate reconstructions using low dimensional encodings. As expected, much higher accuracies were observed using high dimensional encodings, with outline-based methods significantly outperforming image-based autoencoders. The latter tended to encode all cells as having smooth shapes, even for high dimensions. For complex 3D cell shapes, we developed a significant improvement of a method based on the spherical harmonic transform that performs significantly better than other methods. We obtained similar results for the joint modeling of cell and nuclear shape. Finally, we evaluated the modeling of shape dynamics by interpolation in the shape space. We found that our modified method provided lower deformation energies along linear interpolation paths than other methods. This allows practical shape evolution in high dimensional shape spaces. We conclude that our improved spherical harmonic based methods are preferable for cell and nuclear shape modeling, providing better representations, higher computational efficiency and requiring fewer training images than deep learning methods. Availability and implementation: All software and data is available at http://murphylab.cbd.cmu. edu/software. Contact: murphy@cmu.edu Supplementary information: Supplementary data are available at Bioinformatics online.

Supramolecular signaling assemblies are of interest for their unique signaling properties. A µm scale signaling assembly, the central supramolecular signaling cluster (cSMAC), forms at the center of the interface of T cells activated by antigen-presenting cells. We have determined that it is composed of multiple complexes of a supramolecular volume of up to 0.5 µm3 and associated with extensive membrane undulations. To determine cSMAC function, we have systematically manipulated the localization of three adaptor proteins, LAT, SLP-76, and Grb2. cSMAC localization varied between the adaptors and was diminished upon blockade of the costimulatory receptor CD28 and deficiency of the signal amplifying kinase Itk. Reconstitution of cSMAC localization restored IL-2 secretion which is a key T cell effector function as dependent on reconstitution dynamics. Our data suggest that the cSMAC enhances early signaling by facilitating signaling interactions and attenuates signaling thereafter through sequestration of a more limited set of signaling intermediates.

Cellular differentiation is a complex process requiring the coordination of many cellular components. PC12 cells are a popular model system to study changes driving and accompanying neuronal differentiation. While significant attention has been paid to changes in transcriptional regulation and protein signaling, much less is known about the changes in cell organization that accompany PC12 differentiation. Fluorescence microscopy can provide extensive information about this, although photobleaching and phototoxicity frequently limit the ability to continuously observe changes in single cells over the many days that differentiation occurs. Here we describe a generative model of differentiation-associated changes in cell and nuclear shape and their relationship to mitochondrial distribution constructed from images of different cells at discrete time points. We show that our spherical harmonic-based model can accurately represent cell and nuclear shapes by measuring reconstruction errors. We then learn a regression model that relates cell and nuclear shape and mitochondrial distribution and observe that the predictive accuracy generally increases during differentiation. Most importantly, we propose a method, based on cell matching and linear interpolation in the shape space, to model the dynamics of cell differentiation using only static images. Without any prior knowledge, the method produces a realistic shape evolution process.Author Summary Cellular differentiation is an important process that is challenging to study due to the number of organizational changes it includes and the different time scales over which it occurs. Fluorescent microscopy is widely used to study cell dynamics and differentiation, but photobleaching and phototoxicity often make it infeasible to continuously observe a single cell undergoing differentiation for several days. In this work, we described a method to model aspects of the dynamics of PC12 cell differentiation without continuous imaging. We constructed accurate representations of cell and nuclear shapes and quantified the relationships between shapes and mitochondrial distributions. We used these to construct a generative model and combined it with a matching process to infer likely sequences of the changes in single cells undergoing differentiation.

Motivation: Efforts to model how signaling and regulatory networks work in cells have largely either not considered spatial organization or have used compartmental models with minimal spatial resolution. Fluorescence microscopy provides the ability to monitor the spatiotemporal distribution of many molecules during signaling events, but as of yet no methods have been described for large scale image analysis to learn a complex protein regulatory network. Here we present and evaluate methods for identifying how changes in concentration in one cell region influence concentration of other proteins in other regions.Results: Using 3D confocal microscope movies of GFP- tagged T cells undergoing costimulation, we learned models containing putative causal relationships among 12 proteins involved in T cell signaling. The models included both relationships consistent with current knowledge and novel predictions deserving further exploration. Further, when these models were applied to the initial frames of movies of T cells that had been only partially stimulated, they predicted the localization of proteins at later times with statistically significant accuracy. The methods, consisting of spatiotemporal alignment, automated region identification, and causal inference, are anticipated to be applicable to a number of biological systems.

Fluorescence microscopy is one of the most important tools in cell biology research because it provides spatial and temporal information to investigate regulatory systems inside cells. This technique can generate data in the form of signal intensities at thousands of positions resolved inside individual live cells. However, given extensive cell-to-cell variation, these data cannot be readily assembled into three- or four-dimensional maps of protein concentration that can be compared across different cells and conditions. We have developed a method to enable comparison of imaging data from many cells and applied it to investigate actin dynamics in T cell activation. Antigen recognition in T cells by the T cell receptor (TCR) is amplified by engagement of the costimulatory receptor CD28. We imaged actin and eight core actin regulators to generate over a thousand movies of T cells under conditions in which CD28 was either engaged or blocked in the context of a strong TCR signal. Our computational analysis showed that the primary effect of costimulation blockade was to decrease recruitment of the activator of actin nucleation WAVE2 (Wiskott-Aldrich syndrome protein family verprolin-homologous protein 2) and the actin-severing protein cofilin to F-actin. Reconstitution of WAVE2 and cofilin activity restored the defect in actin signaling dynamics caused by costimulation blockade. Thus, we have developed and validated an approach to quantify protein distributions in time and space for the analysis of complex regulatory systems.