Background/Objective: Longitudinal in vivo studies of murine xenograft models are widely utilized in oncology to study cancer biology and develop therapies. Magnetic resonance imaging (MRI) of these tumors is an invaluable tool for monitoring tumor growth and characterizing the tumors as well. Methods: In this work, a pipeline for automating the segmentation of xenografts in mouse models was developed. T2-weighted (T2-wt) MRI images from mice implanted with six different prostate cancer patient-derived xenografts (PDX) in the kidneys, liver, and tibia were used. The segmentation pipeline included a slice classifier to identify the slices that had tumors and subsequent training and validation using several U-Net-based segmentation architectures. Multiple combinations of the algorithm and training images for different sites were evaluated for inference quality. Results and Conclusions: The slice classifier network achieved 90% accuracy in identifying slices containing tumors. Among the various segmentation architectures tested, the dense residual recurrent U-Net achieved the highest performance in kidney tumors. When evaluated across the kidneys, tibia, and liver, this architecture performed the best when trained on all data as compared to training on only data from a single site (and inferring on a multi-site tumor images), achieving a Dice score of 0.924 across the test set.

Non-invasive prostate cancer classification from MRI has the potential to revolutionize patient care by providing early detection of clinically significant disease, but has thus far shown limited positive predictive value. To address this, we present a image-based deep learning method to predict clinically significant prostate cancer from screening MRI in patients that subsequently underwent biopsy with results ranging from benign pathology to the highest grade tumors. Specifically, we demonstrate that mixed supervision via diverse histopathological ground truth improves classification performance despite the cost of reduced concordance with image-based segmentation. Where prior approaches have utilized pathology results as ground truth derived from targeted biopsies and whole-mount prostatectomy to strongly supervise the localization of clinically significant cancer, our approach also utilizes weak supervision signals extracted from nontargeted systematic biopsies with regional localization to improve overall performance. Our key innovation is performing regression by distribution rather than simply by value, enabling use of additional pathology findings traditionally ignored by deep learning strategies. We evaluated our model on a dataset of 973 (testing n = 198) multi-parametric prostate MRI exams collected at UCSF from 2016-2019 followed by MRI/ultrasound fusion (targeted) biopsy and systematic (nontargeted) biopsy of the prostate gland, demonstrating that deep networks trained with mixed supervision of histopathology can feasibly exceed the performance of the Prostate Imaging-Reporting and Data System (PI-RADS) clinical standard for prostate MRI interpretation (71.6% vs 66.7% balanced accuracy and 0.724 vs 0.716 AUC).

BACKGROUND:In modern positron emission tomography (PET) with multi-modality imaging (e.g., PET/CT and PET/MR), the attenuation correction (AC) is the single largest correction factor for image reconstruction. One way to assess AC methods and other reconstruction parameters is to utilize software-based simulation tools, such as a lesion insertion tool. Extensive validation of these simulation tools is required to ensure results of the study are clinically meaningful. PURPOSE:To evaluate different PET AC methods using a synthetic lesion insertion tool that simulates lesions in a patient cohort that has both PET/MR and PET/CT images. To further demonstrate how lesion insertion tool may be used to extend knowledge of PET reconstruction parameters, including but not limited to AC. METHODS:Lesion quantitation is compared using conventional Dixon-based MR-based AC (MRAC) to that of using CT-based AC (CTAC, a "ground truth"). First, the pre-existing lesions were simulated in a similar environment; a total of 71 lesions were identified in 18 pelvic PET/MR patient images acquired with a time-of-flight simultaneous PET/MR scanner, and matched lesions were inserted contralaterally on the same axial slice. Second, synthetic lesions were inserted into four anatomic target locations in a cohort of four patients who didn't have any observed clinical lesions in the pelvis. RESULTS:The matched lesion insertions resulted in unity between the lesion error ratios (mean SUVs), demonstrating that the inserted lesions successfully simulated the original lesions. In the second study, the inserted lesions had distinct characteristics by target locations and demonstrated negative max-SUV%diff trends for bone-dominant sites across the patient cohort. CONCLUSIONS:The current work demonstrates that the applied lesion insertion tool can simulate uptake in pelvic lesions and their expected SUV values, and that the lesion insertion tool can be extended to evaluate further PET reconstruction corrections and algorithms and their impact on quantitation accuracy and precision.

Modern artificial intelligence (AI) and deep learning (DL) techniques have been remarkably successful in predicting Alzheimer's Disease diagnosis and conversion to dementia. However, more continuous outcome measures such as cognitive assessment are needed for richer diagnosis, prognosis, disease trajectory tracking and cohort enrichment in clinical trials. Currently, subjective, time-consuming and operator-sensitive neurocognitive batteries remain the only viable methods of assessing cognition. Very few successful DL studies exist for predicting continuous measures like cognition; those rare ones typically require multiple expensive molecular neuroimaging modalities, fluid biomarkers and multiple visits. Given that MRI is the most prevalent and clinically available imaging modality, its use for quantifying progression and predicting cognition is of great interest. Yet, MRI alone does not capture clinical heterogeneity effectively and has not proven useful in modern DL models of AD progression. Here we propose a novel multitask DL strategy leveraging both domain knowledge and large pretrained models. We experiment with transfer learning using pretrained models in imaging like ResNet50; and customizing domain knowledge-informed loss functions and image-derived latent representations. Our objective is to predict cognitive scores using only baseline MRI and demographics, bypassing the need for longitudinal or multimodal data, neurocognitive assessment or specialized neuroimaging analysis skills -- a more practical goal that can be accomplished in most community clinic settings. Specifically, we show that the latent representations of the patient's MRI tuned to the tissue segmentation task are powerful regularization attributes and excellent input features for the cognition prediction task. This knowledge-based multitask implementation of DL vastly outperforms all existing methods including straightforward transfer learning on large pre-trained models, producing high-quality segmentation maps, accurate diagnosis, and both current and future cognitive scores, using a single MRI and demographic data readily available at baseline. This study has deep implications in early diagnosis, prognosis, and clinical trial design.

Early prostate cancer detection and staging from MRI are extremely challenging tasks for both radiologists and deep learning algorithms, but the potential to learn from large and diverse datasets remains a promising avenue to increase their generalization capability both within- and across clinics. To enable this for prototype-stage algorithms, where the majority of existing research remains, in this paper we introduce a flexible federated learning framework for cross-site training, validation, and evaluation of deep prostate cancer detection algorithms. Our approach utilizes an abstracted representation of the model architecture and data, which allows unpolished prototype deep learning models to be trained without modification using the NVFlare federated learning framework. Our results show increases in prostate cancer detection and classification accuracy using a specialized neural network model and diverse prostate biopsy data collected at two University of California research hospitals, demonstrating the efficacy of our approach in adapting to different datasets and improving MR-biomarker discovery. We open-source our FLtools system, which can be easily adapted to other deep learning projects for medical imaging.

We introduce a compressed sensing technique for leveraging prior electro-optic (EO) imagery to improve 3D synthetic aperture radar (SAR) imaging performance. Specifically, we build on existing iterative reconstruction algorithms by guiding the reconstruction process with a joint-sparsity regularization term that captures the complementary structural information shared between EO and SAR via a sparsifying transform in the 3D image domain. We demonstrate this approach using the wavelet transform, the non-uniform Fast-Fourier transform (NUFFT), and optimizers built on autograd utilizing the 2004 AFRL Gotcha SAR dataset, with complementary EO imagery collected from the 2013 Minor Area Motion Imagery (MAMI) collection and more recent (2016) satellite collections over the same area. Results indicate significant improvements in 2D and 3D imaging performance via incorporation of the cross-modality EO prior, which we attribute to the convex problem formulation.

BACKGROUND:In the initial staging of patients with high-risk prostate cancer (PCa), prostate-specific membrane antigen positron emission tomography (PSMA-PET) has been established as a front-line imaging modality. The increasing number of PSMA-PET scans performed in the primary staging setting might be associated with decreases in biochemical recurrence (BCR)-free survival (BCR-FS). OBJECTIVE:To assess the added prognostic value of presurgical PSMA-PET for BCR-FS compared with the presurgical Cancer of the Prostate Risk Assessment (CAPRA) and postsurgical CAPRA-Surgery (CAPRA-S) scores in patients with intermediate- to high-risk PCa treated with radical prostatectomy (RP) and pelvic lymph node dissection. DESIGN, SETTING, AND PARTICIPANTS:This is a follow-up study of the surgical cohort evaluated in the multicenter prospective phase 3 imaging trial (n = 277; NCT03368547, NCT02611882, and NCT02919111). OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS:Each 68Ga-PSMA-11-PET scan was read by three blinded independent readers. PSMA-PET prostate uptake (low vs high), PSMA-PET extraprostatic disease (N1/M1), and CAPRA and CAPRA-S scores were used to assess the risk of BCR. Patients were followed after RP by local investigators using electronic medical records. BCR was defined by a prostate-specific antigen (PSA) level increasing to ≥0.2 ng/ml after RP or initiation of PCa-specific secondary treatment (>6 mo after surgery). Univariate and multivariable Cox models, and c-statistic index were performed to assess the prognostic value of PSMA-PET and for a comparison with the CAPRA and CAPRA-S scores. RESULTS AND LIMITATIONS:From December 2015 to December 2019, 277 patients underwent surgery after PSMA-PET. Clinical follow-up was obtained in 240/277 (87%) patients. The median follow-up after surgery was 32.4 (interquartile range 23.3-42.9) mo. Of 240 BCR events, 91 (38%) were observed. PSMA-PET N1/M1 was found in 41/240 (17%) patients. PSMA-PET prostate uptake, PSMA-PET N1/M1, and CAPRA and CAPRA-S scores were significant univariate predictors of BCR. The addition of PSMA-PET N1/M1 status to the presurgical CAPRA score improved the risk assessment for BCR significantly in comparison with the presurgical CAPRA score alone (c-statistic 0.70 [0.64-0.75] vs 0.63 [0.57-0.69]; p < 0.001). The C-index of the postsurgical model utilizing the postsurgical CAPRA-S score alone was not significantly different from the presurgical model combining the presurgical CAPRA score and PSMA-PET N1/M1 status (p = 0.19). CONCLUSIONS:Presurgical PSMA-PET was a strong prognostic biomarker improving BCR-FS risk assessment. Its implementation in the presurgical risk assessment with the CAPRA score improved the performance and reduced the difference with the reference standard (postsurgical CAPRA-S score). PATIENT SUMMARY:The use prostate-specific membrane antigen positron emission tomography improved the assessment of biochemical recurrence risk in patients with intermediate- and high-risk prostate cancer who were treated with radical prostatectomy and pelvic lymph node dissection.

Despite the proliferation of deep learning techniques for accelerated MRI acquisition and enhanced image reconstruction, the construction of large and diverse MRI datasets continues to pose a barrier to effective clinical translation of these technologies. One major challenge is in collecting the MRI raw data (required for image reconstruction) from clinical scanning, as only magnitude images are typically saved and used for clinical assessment and diagnosis. The image phase and multi-channel RF coil information are not retained when magnitude-only images are saved in clinical imaging archives. Additionally, preprocessing used for data in clinical imaging can lead to biased results. While several groups have begun concerted efforts to collect large amounts of MRI raw data, current databases are limited in the diversity of anatomy, pathology, annotations, and acquisition types they contain. To address this, we present a method for synthesizing realistic MR data from magnitude-only data, allowing for the use of diverse data from clinical imaging archives in advanced MRI reconstruction development. Our method uses a conditional GAN-based framework to generate synthetic phase images from input magnitude images. We then applied ESPIRiT to derive RF coil sensitivity maps from fully sampled real data to generate multi-coil data. The synthetic data generation method was evaluated by comparing image reconstruction results from training Variational Networks either with real data or synthetic data. We demonstrate that the Variational Network trained on synthetic MRI data from our method, consisting of GAN-derived synthetic phase and multi-coil information, outperformed Variational Networks trained on data with synthetic phase generated using current state-of-the-art methods. Additionally, we demonstrate that the Variational Networks trained with synthetic k-space data from our method perform comparably to image reconstruction networks trained on undersampled real k-space data.

5088 Background: To assess the predictive value of pre-operative PSMA-PET staging for biochemical recurrence (BCR) free-survival (BCR-FS) in patients treated with radical prostatectomy (RP) and pelvic lymph node dissection (PLND) with intermediate-risk (IR) to high-risk (HR) prostate cancer (PCa) included in the prospective trial used for the FDA approval of 68Ga-PSMA-11. Methods: This is a post-hoc follow-up study of the efficacy analysis cohort included in the multicenter prospective phase 3 imaging trial (n = 764; NCT03368547, NCT02611882, NCT02919111) which assessed the diagnostic accuracy of 68Ga-PSMA-11 PET for pelvic nodal metastasis detection prior to RP and PLND in patients with IR and HR PCa. Each PSMA-PET scan was read by three blinded independent readers. Readers assessed the presence of PCa (positive vs negative) by region: prostate bed (T), pelvic lymph nodes (N), extra-pelvic lymph nodes (M1a) bone (M1b) and visceral (M1c). A centralized per-region majority rule was used in case of disagreement. The surgical pathology report was used to assess the presence of pelvic lymph node metastasis by histopathology (pN0 vs pN1). The patients were followed up for biochemical progression after RP by the local investigators using electronic medical records. BCR was defined by a prostate-specific antigen (PSA) level > 0.2 ng/ml after RP or an initiation of PCa specific adjuvant/salvage therapy. Pairwise comparisons using Log-Rank test was performed to evaluate BCR-FS between the pre-operative PSMA scan reads (N0M0 vs. N+ and/or M+) and the histopathology status (pN0 vs. pN1). Results: From December 2015 to December 2019, a total of 764 patients were enrolled in the trial. 277/764 (36%) underwent RP after PSMA-PET. Clinical follow-up was obtained in 240/277 (87%) patients. The median age was 67 years (interquartile range, 61-71 years). The median follow-up time from RP was 21.4 months (IQR: 8.80 - 31.53). One hundred BCR events (41%) were observed, and 98/240 patients underwent salvage therapy or other treatment (40.6%). The BCR-FS was 24.3 (IQR: 7.8 - 48.8) in the whole cohort. 160/240 (66%), 28/240 (11.6%), 39/240 (16%), 13/240 (5.4%) patients were pN0/PSMA- (N0 and M0), pN+/PSMA+ (N+ and/or M+), pN+/PSMA-, and pN0/PSMA+, respectively. BCR-FS was higher in PSMA- than in PSMA+ patients (33 vs 7.3 months; p < 0.0001). BCR-FS was higher in pN0/PSMA- than in patients pN+/PSMA-, pN0/PSMA+ and pN+/PSMA+: 46 months vs 12.3, 11.7, and 3, respectively (p < 0.001). BCR-FS did not significantly differ between pN0/PSMA+ and pN+/PSMA- (11.7 vs 12.3; p = 0.64). Conclusions: PSMA PET staging information is predictive of BCR-FS after RP. Patients with extra-prostatic disease detected by pre-operative PSMA-PET scan have a high risk of biochemical relapse. Clinical trial information: NCT03368547.

In this paper, we review physics- and data-driven reconstruction techniques for simultaneous positron emission tomography (PET) / magnetic resonance imaging (MRI) systems, which have significant advantages for clinical imaging of cancer, neurological disorders, and heart disease. These reconstruction approaches utilize priors, either structural or statistical, together with a physics-based description of the PET system response. However, due to the nested representation of the forward problem, direct PET/MRI reconstruction is a nonlinear problem. We elucidate how a multi-faceted approach accommodates hybrid data- and physics-driven machine learning for reconstruction of 3D PET/MRI, summarizing important deep learning developments made in the last 5 years to address attenuation correction, scattering, low photon counts, and data consistency. We also describe how applications of these multi-modality approaches extend beyond PET/MRI to improving accuracy in radiation therapy planning. We conclude by discussing opportunities for extending the current state-of-the-art following the latest trends in physics- and deep learning-based computational imaging and next-generation detector hardware.

PET/MRI scanners cannot be qualified in the manner adopted for hybrid PET/CT devices. The main hurdle with qualification in PET/MRI is that attenuation correction (AC) cannot be adequately measured in conventional PET phantoms because of the difficulty in converting the MR images of the physical structures (e.g., plastic) into electron density maps. Over the last decade, a plethora of novel MRI-based algorithms has been developed to more accurately derive the attenuation properties of the human head, including the skull. Although promising, none of these techniques has yet emerged as an optimal and universally adopted strategy for AC in PET/MRI. In this work, we propose a path for PET/MRI qualification for multicenter brain imaging studies. Specifically, our solution is to separate the head AC from the other factors that affect PET data quantification and use a patient as a phantom to assess the former. The emission data collected on the integrated PET/MRI scanner to be qualified should be reconstructed using both MRI- and CT-based AC methods, and whole-brain qualitative and quantitative (both voxelwise and regional) analyses should be performed. The MRI-based approach will be considered satisfactory if the PET quantification bias is within the acceptance criteria specified here. We have implemented this approach successfully across 2 PET/MRI scanner manufacturers at 2 sites.

Background Megavoltage computed tomography (MVCT) has been implemented on many radiotherapy treatment machines for on-board anatomical visualization, localization, and adaptive dose calculation. Implementing an MR-only workflow by synthesizing MVCT from magnetic resonance imaging (MRI) would offer numerous advantages for treatment planning and online adaptation. Purpose In this work, we sought to synthesize MVCT (sMVCT) datasets from MRI using deep learning to demonstrate the feasibility of MRI-MVCT only treatment planning. Methods MVCTs and T1-weighted MRIs for 120 patients treated for head-and-neck cancer were retrospectively acquired and co-registered. A deep neural network based on a fully-convolutional 3D U-Net architecture was implemented to map MRI intensity to MVCT HU. Input to the model were volumetric patches generated from paired MRI and MVCT datasets. The U-Net was initialized with random parameters and trained on a mean absolute error (MAE) objective function. Model accuracy was evaluated on 18 withheld test exams. sMVCTs were compared to respective MVCTs. Intensity-modulated volumetric radiotherapy (IMRT) plans were generated on MVCTs of four different disease sites and compared to plans calculated onto corresponding sMVCTs using the gamma metric and dose-volume-histograms (DVHs). Results MAE values between sMVCT and MVCT datasets were 93.3 +/- 27.5, 78.2 +/- 27.5, and 138.0 +/- 43.4 HU for whole body, soft tissue, and bone volumes, respectively. Overall, there was good agreement between sMVCT and MVCT, with bone and air posing the greatest challenges. The retrospective dataset introduced additional deviations due to sinus filling or tumor growth/shrinkage between scans, differences in external contours due to variability in patient positioning, or when immobilization devices were absent from diagnostic MRIs. Dose distributions of IMRT plans evaluated for four test cases showed close agreement between sMVCT and MVCT images when evaluated using DVHs and gamma dose metrics, which averaged to 98.9 +/- 1.0% and 96.8 +/- 2.6% analyzed at 3%/3 mm and 2%/2 mm, respectively. Conclusions MVCT datasets can be generated from T1-weighted MRI using a 3D deep convolutional neural network with dose calculation on a sample sMVCT in close agreement with the MVCT. These results demonstrate the feasibility of using MRI-derived sMVCT in an MR-only treatment planning workflow.

Historically, patient datasets have been used to develop and validate various reconstruction algorithms for PET/MRI and PET/CT. To enable such algorithm development, without the need for acquiring hundreds of patient exams, in this article we demonstrate a deep learning technique to generate synthetic but realistic whole-body PET sinograms from abundantly available whole-body MRI. Specifically, we use a dataset of 56 18F-FDG-PET/MRI exams to train a 3-D residual UNet to predict physiologic PET uptake from whole-body T1-weighted MRI. In training, we implemented a balanced loss function to generate realistic uptake across a large dynamic range and computed losses along tomographic lines of response to mimic the PET acquisition. The predicted PET images are forward projected to produce synthetic PET (sPET) time-of-flight (ToF) sinograms that can be used with vendor-provided PET reconstruction algorithms, including using CT-based attenuation correction (CTAC) and MR-based attenuation correction (MRAC). The resulting synthetic data recapitulates physiologic 18F-FDG uptake, e.g., high uptake localized to the brain and bladder, as well as uptake in liver, kidneys, heart, and muscle. To simulate abnormalities with high uptake, we also insert synthetic lesions. We demonstrate that this sPET data can be used interchangeably with real PET data for the PET quantification task of comparing CTAC and MRAC methods, achieving ≤ 7.6% error in mean-SUV compared to using real data. These results together show that the proposed sPET data pipeline can be reasonably used for development, evaluation, and validation of PET/MRI reconstruction methods.

We present a new strategy for incorporating high resolution structural information from MRI into the reconstruction of PET imagery via deep domain translated image priors. The strategy involves two steps: (1) predicting a PET uptake volume directly from MRI without requiring a radiation dose, and (2) using the predicted dose-free PET volume to impose sparsity constraints on the PET reconstruction from measured sinograms. The key idea of our approach is that domain translated PET imagery can capture the true spatial and sparsity patterns of PET imagery, which can be used to guide the convergence of the statistics-limited inverse problem. This scheme can be superior to joint-sparsity reconstruction, among other methods, since the mismatch between PET and MRI features is significantly reduced by using the domain translated zero-dose PET as the prior instead. We evaluate this technique on a wholebody 18F-FDG-PET dataset, demonstrating that dichromatic interpolation can recover high quality PET imagery from noisy and low dose PET/MRI, with no observed failure cases.

ABSTRACT During its chronic degenerative course, Alzheimer’s Disease severely harms the patients’ cognitive abilities. Assessment of current and future cognition is an integral component of a diagnosis of dementia, and therefore an important clinical and scientific goal. Unfortunately, subjective, time-consuming and operator-sensitive clinical surveys or neuropyschiatric batteries remain the only viable methods of assessing cognition. Given that MRI is the most prevalent, cost-effective, and clinically important imaging modality, it may be considered a suitable predictor of cognition. Yet, it has hitherto proved very challenging to predict one from the other. We propose that an image-based Deep Learning model can be custom-built to achieve this goal. We designed a novel multi-task UNet model to predict the subjects’ current and future cognition (via ADAS-Cog scores), taking as input baseline T1-weighted MRI and demographic risk factors. The key innovation in the model is that it seeks to solve two adjacent but relevant tasks: image segmentation into tissue types; and prediction of cognition. The first task gives a high-accuracy brain segmentation, comparable to other cutting edge methods. The features trained from the segmentation task are used in the cognition task. This combination is far superior to stand-alone single-shot cognition models. We achieved excellent accuracy in both baseline and time-series forecast of ADAS-Cog scores. Through further feature map analysis made on the receptive fields, we managed to impart much-needed model interpretability, critical for real-world clinical practice. This study constitutes the best-reported performance of any comparable approach, and opens the door towards machine-based tracking of AD progression.

Objective: Simultaneous PET/MRIs vary in their quantitative PET performance due to inherent differences in the physical systems and differences in the image reconstruction implementation. This variability in quantitative accuracy confounds the ability to meaningfully combine and compare data across scanners. In this work, we define image reconstruction hyperparameters that lead to comparable contrast recovery curves across simultaneous PET/MRI systems.Method: The NEMA NU-2 image quality phantom was imaged on one GE Signa and on one Siemens mMR PET/MRI scanner. The phantom was imaged at 9.8:1 contrast with standard spheres (diameter 10, 13, 17, 22, 28, 37mm) and with custom spheres (diameter: 8.5, 11.5, 15, 25, 32.5, 44 mm) using a standardized methodology. Analysis was performed on a 30 minute listmode data acquisition and on 6 realizations of 5 minutes from the listmode data. Images were reconstructed with the manufacturer provided iterative image reconstruction algorithms with and without point spread function (PSF) modeling. For both scanners, a post-reconstruction Gaussian filter of 3 to 7 mm in steps of 1 mm were applied. Attenuation correction was provided from a scaled Computed Tomography (CT) image of the phantom registered to the MR-based attenuation images and verified to align on the non-attenuated corrected PET images. For each of these image reconstruction parameter sets, contrast recovery coefficients (CRCs) were determined for the SUVmean, SUVmax and SUVpeak for each sphere. A hybrid metric combining the root mean squared discrepancy (RMSD) and the absolute CRC values were used to simultaneously optimize for best match in CRC between the two scanners while simultaneously weighting towards higher resolution reconstructions. The image reconstruction hyperparameter set were identified as the best candidate reconstruction for each vendor for harmonized PET image reconstruction. Results: The range of clinically relevant image reconstruction hyperparameters demonstrated widely different quantitative performance across cameras. The best match of CRC curves were obtained at the lowest RMSD values with: for CRCmean, 2 iterations -7mm filter with PSF on the GE Signa and 4 iterations -6mm filter on the Siemens mMR, for CRCmax, 2 iterations -7mm filter on the GE Signa, 4 iterations - 6mm filter on the Siemens mMR and for CRCeak, 4 iterations-7mm filter with PSF on the GE Signa and 3 iterations-6mm filter on the Siemens mMR. Over all reconstructions, the RMSD between CRCs were 2.4%, 3.1% and 2.3% for CRC mean, max and peak, respectively. The solution of 2 iterations-3mm on the GE Signa and 4 iterations-3mm on Siemens mMR, both with PSF, led to simultaneous harmonization and with high CRC and low RMSD for CRC mean, max and peak with RMSD values of 3.4%, 5.5% and 3.0 %, respectively.Conclusions: For two commercially-available PET/MRI scanners, user-selectable hyperparameters that control iterative updates, image smoothing, and PSF-modeling provide a range of contrast recovery curves that allow harmonization in harmonization strategies of optimal match in CRC or high CRC values. This work demonstrates that nearly identical CRC curves can be obtained on different commercially available scanners by selecting appropriate image reconstruction hyperparameters.

Fully-convolutional neural networks, such as the 2D or 3D UNet, are now pervasive in medical imaging for semantic segmentation, classification, image denoising, domain translation, and reconstruction. However, evaluation of UNet performance, as with most CNNs, has mostly been relegated to evaluation of a few performance metrics (e.g. accuracy, IoU, SSIM, etc.) using the network's final predictions, which provides little insight into important issues such as dataset shift that occur in clinical application. In this paper, we propose techniques for understanding and visualizing the generalization performance of UNets in image classification and regression tasks, giving rise to metrics that are indicative of performance on a withheld test-set without the need for groundtruth annotations.

In this paper, we present a pipeline and prototype vision system for near-real-time semantic segmentation and classification of objects such as roads, buildings, and vehicles in large high-resolution wide-area real-world aerial LiDAR point-cloud and RGBD imagery. Unlike previous works, which have focused on exploiting ground-based sensors or narrowed the scope to detecting the density of large objects, here we address the full semantic segmentation of aerial LiDAR and RGBD imagery by exploiting crowd-sourced labels that densely canvas each image in the 2015 Dublin dataset.(1) Our results indicate important improvements to detection and segmentation accuracy with the addition of aerial LiDAR over RGB imagery alone, which has important implications for civilian applications such as autonomous navigation and rescue operations. Moreover, the prototype system can segment and search geographic areas as big as 1km(2) in a matter of seconds on commodity hardware with high accuracy (>= 90%), suggesting the feasibility of real-time scene understanding on small aerial platforms. [GRAPHICS] .

We study generalization in deep learning by appealing to complexity measures originally developed in approximation and information theory. While these concepts are challenged by the high-dimensional and data-defined nature of deep learning, we show that simple vector quantization approaches such as PCA, GMMs, and SVMs capture their spirit when applied layer-wise to deep extracted features giving rise to relatively inexpensive complexity measures that correlate well with generalization performance. We discuss our results in 2020 NeurIPS PGDL challenge.