Image-computable models of primate retinal ganglion cell (RGC) mosaics that are synthesized and constrained jointly by optical, anatomical and physiological properties, and which operate on images defined by their spatial-spectral radiance, do not currently exist. Here, we deploy a novel computational framework which synthesizes mosaics of linear spatio-chromatic receptive fields (RFs) of ON midget RGCs (mRGCs) by integrating published anatomical, physiological, and optical quality measurements, all varying with eccentricity. We use the synthesized mRGC mosaics to simulate both in vivo and in vitro physiological experiments and demonstrate the model’s consistency with published data. The model enables computation of how visual performance is shaped by the representation of visual information provided by the linear spatiochromatic processing stage of midget RGCs. The developed computational framework carefully accounts for the effect of physiological optics on mRGC responses, enables comparison of in vivo and in vitro data, and allows exploration of how different assumptions about RF organization, such as selectivity for the type of cones pooled by the RF center mechanism, affect physiological responses and psychophysical performance. The open-source and freely available implementation provides a platform for understanding how the linear spatiochromatic receptive field representation of the mRGCs shapes visual performance, as well as a foundation for future work that incorporates response nonlinearities, temporal filtering, and extends to additional RGC mosaics.

Accurate image-computable models of retinal ganglion cell (RGC) mosaics across the retina do not currently exist. Here, we deploy a novel computational framework which synthesizes mosaics of linear spatio-chromatic receptive fields (RFs) of ON midget RGCs (mRGCs) by integrating published anatomical, physiological, and optical quality measurements. We use the synthesized mRGC mosaics to simulate both in vivo and in vitro physiological experiments and demonstrate the model’s consistency with published data. The model enables computation of how visual performance is shaped by the representation of visual information provided by the linear spatiochromatic processing stage of midget RGCs. The developed computational framework carefully accounts for the effect of physiological optics on mRGC responses, enables comparison of in vivo and in vitro data, and allows exploration of how different assumptions about RF organization, such as selectivity for the type of cones pooled by the RF center mechanism, affect physiological responses and psychophysical performance. The open-source and freely available implementation provides a platform for understanding how the linear spatiochromatic receptive field representation of the mRGCs shapes visual performance, as well as a foundation for future work that incorporates response nonlinearities, temporal filtering, and extends to additional RGC mosaics. Author summary We present a comprehensive, image-computable model of the human midget retinal ganglion cell (mRGC) mosaic that integrates diverse anatomical, optical, and physiological data. A central challenge in retinal modeling is reconciling measurements from in vivo recordings, which are affected by the eye’s optics, and in vitro recordings, which are not. Our model overcomes this by explicitly separating the optical and post-receptoral stages of visual processing. The synthesis of the mRGC receptive fields is guided by multiple constraints, including cone and mRGC densities, macaque neurophysiology, and human optical quality. The resulting model validates well against both in vivo and in vitro data and provides a powerful tool for understanding how linear spatial pooling in the retina limits human visual performance across the visual field. ### Competing Interest Statement The authors have declared no competing interest.

The human visual system is a network of neural components that combine to create our perception of the world and guide our behavior. Deciphering the computational principles of this system is an important scientific challenge. We review measurements of these components, from the retinal encoding to cortical circuitry, and from molecules to circuits, focusing on measurements that are relevant to visual processing. We then delve into principles proposed to explain how this diverse collection of visual components enables us to interpret our surroundings.

This paper describes a physics-based end-to-end software simulation for image systems. We use the software to explore sensors designed to enhance performance in high dynamic range (HDR) environments, such as driving through daytime tunnels and under nighttime conditions. We synthesize physically realistic HDR spectral radiance images and use them as the input to digital twins that model the optics and sensors of different systems. This paper makes three main contributions: (a) We create a labeled (instance segmentation and depth), synthetic radiance dataset of HDR driving scenes. (b) We describe the development and validation of the end-to-end simulation framework. (c) We present a comparative analysis of two single-shot sensors designed for HDR. We open-source both the dataset and the software.

This study explores the potential of quantitative autofluorescence imaging (AFI) as an objective tool for monitoring the health of oral mucosal tissue. Our approach involves acquiring spectroradiometric measurements of tissue fluorescence and utilizing a model to understand the fluorophores influencing these measurements, including the impact of blood attenuation. We acquired fluorescence measurements from the dorsal tongue and inner lip of healthy human volunteers, subsequently fitting the model to these data to estimate individual fluorophore contributions and the optical density of the blood. Our dataset and model, which are freely shared in an open repository, aim to advance the development of quantitative, non-invasive diagnostic imaging systems for monitoring oral health, ultimately facilitating the detection and characterization of oral mucosal tissue abnormalities. ### Competing Interest Statement The authors have declared no competing interest.

Spectroradiometric fluorescence measurements were collected from the dorsal tongue and inner lip of healthy volunteers. These sites were chosen to represent the distinct spectral features that differentiate keratinized from non-keratinized oral tissues, as documented in previous studies. A computational model was then applied to estimate the relative contributions of key fluorophores and to quantify the influence of blood absorption on the observed fluorescence spectra. The resulting dataset and model, both freely available, serve as reference standards for healthy oral tissue and support the development of quantitative, non-invasive imaging systems for consistent and reproducible assessment of oral mucosal health.

Two ideas, proposed by Thomas Young and James Clerk Maxwell, form the foundations of colour science: (i) three types of retinal receptors encode light under daytime conditions, and (ii) colour matching experiments establish the critical spectral properties of this encoding. Experimental quantification of these ideas is used in international colour standards. However, for many years, the field did not reach consensus on the spectral properties of the biological substrate of colour matching: the spectral sensitivity of the cone fundamentals. By combining auxiliary data (thresholds, inert pigment analyses), complex calculations, and colour matching from genetically analysed dichromats, the human cone fundamentals have now been standardized. Here, we describe a new computational method to estimate the cone fundamentals using only colour matching from the three types of dichromatic observers. We show that it is not necessary to include data from trichromatic observers in the analysis or to know the primary lights used in the matching experiments. Remarkably, it is even possible to estimate the fundamentals by combining data from experiments using different, unknown primaries. We then suggest how the new method may be applied to colour management in modern image systems.

A 2D-line-scan MRI sequence has been reported to directly measure neural responses to stimuli (the "DIANA response"). Attempts to replicate the DIANA response have failed, even with higher field strength and more repetitions. Part of this discrepancy is likely due to a limited understanding of how physiological noise manifests in 2D-line-scan acquisition sequences. Specifically, it is unclear what the consequences are of breaking the assumption that the imaging substrate remains constant between each line acquisition. To answer this question, we collected 2D-line-scan data at 3T from human subjects viewing a blank screen. We found temporal fluctuations in the reconstructed time series that could easily be confused with neural responses to stimuli. These fluctuations were present both in the head and in the surrounding empty volume along the span of the phase-encoding direction from the head. The timing of these fluctuations varied systematically and smoothly along the phase-encoding direction. These artifacts are similar to well-known phase-encoding artifacts in EPI and GRE images, but are exacerbated due to longer acquisition times in the 2D-line-scan sequence (seconds vs. milliseconds). We explain these artifacts with a model that accounts for the acquisition sequence and incorporates time-varying contrast fluctuations and movement in the imaging substrate. Using the model, we quantify the amount of cortical- and scan-averaging one might need to reliably distinguish a DIANA response from noise, and show that navigator echoes might help in reducing phase-encode noise in the 2D-line-scan sequence.

The design and evaluation of complex systems can benefit from a software simulation - sometimes called a digital twin. The simulation can be used to characterize system performance or to test its performance under conditions that are difficult to measure (e.g., nighttime for automotive perception systems). We describe the image system simulation software tools that we use to evaluate the performance of image systems for object (automobile) detection. We describe experiments with 13 different cameras with a variety of optics and pixel sizes. To measure the impact of camera spatial resolution, we designed a collection of driving scenes that had cars at many different distances. We quantified system performance by measuring average precision and we report a trend relating system resolution and object detection performance. We also quantified the large performance degradation under nighttime conditions, compared to daytime, for all cameras and a COCO pre-trained network.

Diffusion MRI is a complex technique, where new discoveries and implementations occur at a fast pace. The expertise needed for data analyses and accurate and reproducible results is increasingly demanding and requires multidisciplinary collaborations. In the present work we introduce Reproducible Tract Profiles 2 (RTP2), a set of flexible and automated methods to analyze anatomical MRI and diffusion weighted imaging (DWI) data for reproducible tractography. RTP2 reads structural MRI data and processes them through a succession of serialized containerized analyses. We describe the DWI algorithms used to identify white-matter tracts and their summary metrics, the flexible architecture of the platform, and the tools to programmatically access and control the computations. The combination of these three components provides an easy-to-use automatized tool developed and tested over 20 years, to obtain usable and reliable state-of-the-art diffusion metrics at the individual and group levels for basic research and clinical practice.

Vision science combines ideas from physics, biology, and psychology. The language and ideas of mathematics help scientists communicate and provide an initial framing for understanding the visual system. Mathematics combined with computational modeling adds important realism to the formulations. Together, mathematics and computational tools provide a realistic estimate of the initial signals the brain analyzes to render visual judgments (e.g., motion, depth, and color). This chapter first traces calculations from the representation of the light signal, to how that signal is transformed by the lens to the retinal image, and then how the image is converted into cone photoreceptor excitations. The central steps in the initial encoding rely heavily on linear systems theory and the mathematics of signal-dependent noise. We then describe computational methods that add more realism to the description of how light is encoded by cone excitations. Finally, we describe the mathematical formulation of the ideal observer using all the encoded information to perform a visual discrimination task, and Bayesian methods that combine prior information and sensory data to estimate the light input. These tools help us reason about the information present in the neural representation, what information is lost, and types of neural circuits for extracting information.

Blood and cerebrospinal fluid (CSF) pulse and flow throughout the brain, driven by the cardiac cycle. These fluid dynamics, which are essential to healthy brain function, are characterized by several noninvasive magnetic resonance imaging (MRI) methods. Recent developments in fast MRI, specifically simultaneous multislice acquisition methods, provide a new opportunity to rapidly and broadly assess cardiac‐driven flow, including CSF spaces, surface vessels and parenchymal vessels. We use these techniques to assess blood and CSF flow dynamics in brief (3.5 min) scans on a conventional 3 T MRI scanner in five subjects. Cardiac pulses are measured with a photoplethysmography (PPG) on the index finger, along with functional MRI (fMRI) signals in the brain. We, retrospectively, align the fMRI signals to the heartbeat. Highly reliable cardiac‐gated fMRI temporal signals are observed in CSF and blood on the timescale of one heartbeat (test–retest reliability within subjects R2 > 50%). In blood vessels, a local minimum is observed following systole. In CSF spaces, the ventricles and subarachnoid spaces have a local maximum following systole instead. Slower resting‐state scans with slice timing, retrospectively, aligned to the cardiac pulse, reveal similar cardiac‐gated responses. The cardiac‐gated measurements estimate the amplitude and phase of fMRI pulsations in the CSF relative to those in the arteries, an estimate of the local intracranial impedance. Cardiac aligned fMRI signals can provide new insights about fluid dynamics or diagnostics for diseases where these dynamics are important.

The midget retinal ganglion cell (mRGC) mosaic forms a critical neural substrate for human pattern and color vision. To help understand mRGCs, we are developing an image-computable model of their spatial receptive fields (RFs) across the central primate retina. The open-source model explicitly incorporates the eye’s optics (including chromatic aberration), spatial and spectral sampling by the interleaved trichromatic cone mosaic, and spatial pooling of cone signals. The mRGC mosaic is synthesized as follows. First, synthetic lattices of cone positions and mRGC RF positions are generated independently, based on anatomical estimates of cone density (Packer et al., 1989) and mRGC RF positions (Watson 2014), using an iterative algorithm (Cottaris et al., 2019). Next, cones are connected to RF centers in a way that optimizes a tradeoff between spatial compactness and spectral homogeneity of the RF center. Finally, cones are pooled by mRGC surrounds with spatial weights based on H1 horizontal cell RF data (Packer & Dacey 2002), optimized to yield visual-field spatial transfer functions (STFs) that approximate those recoded in the macaque (Croner & Kaplan 1995). Point spread functions derived from wavefront-aberration measurements (Polans et al., 2015) are used to link visual field and retinal extents. To validate the model, we fit STFs (computed from the model's responses to drifting achromatic gratings viewed through physiological optics), with a Difference of Gaussians RF model and compare the parameters to the same model fit to measurements obtained in vivo in the macaque (C&K, 1995). The ratio of surround/center radius and surround/center integrated sensitivity agrees closely (mean/std z-score: 0.02 +/- 0.94 and -0.08 +/1.16, respectively). The model center sizes are slightly smaller than those from macaque (mean/std z-score: -1.99, 1.70), which may be due to uncertainty about the optics. Our model offers an image-computable approach for assessing how mRGCs impact spatial and chromatic vision.

We assess the accuracy of a smartphone camera simulation. The simulation is an end-to-end analysis that begins with a physical description of a high dynamic range 3D scene and includes a specification of the optics and the image sensor. The simulation is compared to measurements of a physical version of the scene. The image system simulation accurately matched measurements of optical blur, depth of field, spectral quantum efficiency, scene inter-reflections, and sensor noise. The results support the use of image systems simulation methods for soft prototyping cameras and for producing synthetic data in machine learning applications.

Summary Diffusion MRI is a complex technique, where new discoveries and implementations occur at a fast pace. The expertise needed for data analyses and accurate and reproducible results is increasingly demanding and requires multidisciplinary collaborations. In the present work we introduce Reproducible Tract Profiles (RTP2): a set of flexible and automated methods to analyze anatomical MRI and diffusion weighted imaging (DWI) data for reproducible tractography. The tools read structural MRI data and process them through a succession of serialized containerized analyses. We describe the DWI algorithms used to identify white-matter tracts and their summary metrics, the flexible architecture of the platform, and the tools to programmatically access and control the computations. The combination of these three components provides an easy-to-use automatized tool developed and tested over 20 years, to obtain usable and reliable state-of-the-art diffusion metrics at the individual and group levels for basic research and clinical practice. Highlights Automated, flexible and reproducible protocol for white-matter tractography and tractometry. High computational (same data, different computations) and test-retest reproducibility (data from different sessions, different computations). Open-source code and publicly available containers.

For more than two centuries scientists and engineers have worked to understand and model how the eye encodes electromagnetic radiation (light). We now understand the principles of how light is transmitted through the optics of the eye and encoded by retinal photoreceptors and light-sensitive neurons. In recent years, new instrumentation has enabled scientists to measure the specific parameters of the optics and photoreceptor encoding. We implemented the principles and parameter estimates that characterize the human eye in an open-source software toolbox. This chapter describes the principles behind these tools and illustrates how to use them to compute the initial visual encoding.