Accurate redshift estimates are a vital component in understanding galaxy evolution and precision cosmology. In this paper, we explore approaches to increase the applicability of machine learning models for photometric redshift estimation on a broader range of galaxy types. Typical models are trained with ground-truth redshifts from spectroscopy. We test the utility and effectiveness of two approaches for combining spectroscopic redshifts and redshifts derived from multiband (∼35 filters) photometry, which sample different types of galaxies compared to spectroscopic surveys. The two approaches are (1) training on a composite dataset, and (2) transfer learning from one dataset to another. We compile photometric redshifts from the COSMOS2020 catalog (TransferZ) to complement an established spectroscopic redshift dataset (GalaxiesML). We used two architectures, deterministic neural networks (NN) and Bayesian neural networks (BNN), to examine and evaluate their performance with respect to the Legacy Survey of Space and Time (LSST) photo- z science requirements. We also use split conformal prediction for calibrating uncertainty estimates and producing prediction intervals for the BNN and NN, respectively. We find that a NN trained on a composite dataset predicts photo- z 's 4.5 times less biased within the redshift range 0.3 < z < 1.5, 1.1 times less scattered, and has a 1.4 times lower outlier rate than a model trained on only spectroscopic ground truths. We also find that BNNs produce reliable uncertainty estimates, but are sensitive to the different ground truths. This investigation leverages different sources of ground truths to develop models that can accurately predict photo- z 's for a broader galaxy population, which is crucial for surveys such as Euclid and LSST.

The origin of the radio synchrotron background (RSB) is currently unknown. Its understanding might have profound implications in fundamental physics or might reveal a new class of radio emitters. In this work, we consider the scenario in which the RSB is due to extragalactic radio sources and measure the angular cross-correlation of LOFAR images of the diffuse radio sky with matter tracers at different redshifts, provided by galaxy catalogs and CMB lensing. We compare these measured cross-correlations to those expected for models of RSB sources. We find that low-redshift populations of discrete sources are excluded by the data, while higher redshift explanations are compatible with available observations. We also conclude that at least 20% of the RSB surface brightness level must originate from populations tracing the large-scale distribution of matter in the universe, indicating that at least this fraction of the RSB is of extragalactic origin. Future measurements of the correlation between the RSB and tracers of high-redshift sources will be crucial to constraining the source population of the RSB.

In this work, we explore methods to improve galaxy redshift predictions by combining different ground truths. Traditional machine learning models rely on training sets with known spectroscopic redshifts, which are precise but only represent a limited sample of galaxies. To make redshift models more generalizable to the broader galaxy population, we investigate transfer learning and directly combining ground truth redshifts derived from photometry and spectroscopy. We use the COSMOS2020 survey to create a dataset, TransferZ, which includes photometric redshift estimates derived from up to 35 imaging filters using template fitting. This dataset spans a wider range of galaxy types and colors compared to spectroscopic samples, though its redshift estimates are less accurate. We first train a base neural network on TransferZ and then refine it using transfer learning on a dataset of galaxies with more precise spectroscopic redshifts (GalaxiesML). In addition, we train a neural network on a combined dataset of TransferZ and GalaxiesML. Both methods reduce bias by ∼ 5x, RMS error by ∼ 1.5x, and catastrophic outlier rates by 1.3x on GalaxiesML, compared to a baseline trained only on TransferZ. However, we also find a reduction in performance for RMS and bias when evaluated on TransferZ data. Overall, our results demonstrate these approaches can meet cosmological requirements.

In the emerging era of big data astrophysics, large-scale extragalactic surveys will soon provide high-quality imaging for billions of celestial objects to answer major questions in astrophysics such as the nature of dark matter and dark energy. Precision cosmology with surveys requires accurate photometric redshift (photo-z) estimation with well-constrained uncertainties as inputs for weak lensing models to measure cosmological parameters. Machine learning methods have shown promise in optimizing the information gained from galaxy images in photo-z estimation; however, many of these methods are limited in their ability to estimate accurate uncertainties. In this work, we present one of the first applications of Bayesian convolutional neural networks (BCNNs) for photo-z estimation and uncertainties. In addition, we use conformal mapping to calibrate the photo-z uncertainties to achieve good statistical coverage. We use the public GalaxiesML data set of similar to 300k galaxies from the Hyper Suprime-Cam survey containing five-band photometric images and known spectroscopic redshifts from 0 < z < 4. We find that the performance is much improved when using images compared to photometry, with the BCNN achieving 0.098 rms error, a standard outlier rate of 3.9%, a 3 sigma outlier rate of 4.5%, and a bias of 0.0007. The performance drops significantly beyond z > 1.5 due to the relative lack of training data beyond those redshifts. This investigation demonstrates the power of using images directly and we advocate that future photo-z analysis of large-scale surveys include galaxy images.

Abstract We present SpyderZ, a Python-based library for photometric redshift estimation using support vector machines (implemented with scikit-learn). Our approach discretizes redshift values into uniformly-sized bins and uses one-vs-one support vector classifiers with voting strategies to produce effective probability density functions (ePDFs) over redshift for each galaxy. These ePDFs, which are not constrained to be Gaussian or any other shape, allow for our model's predictions to be used quantitatively with uncertainty analysis methods, and have been shown to enable reliable catastrophic outlier detection. Adapted from the previous IDL package SpiderZ, SpyderZ offers training and evaluation speed optimizations on the order of 102, along with support for parallelization across CPU cores. Our library also offers in-built data sanity checks, result visualizations, metric calculations, cross validation, batch evaluations, and parallelized hyperparameter search (grid search and random search).

We present results exploring the role that probabilistic deep learning models can play in cosmology from large scale astronomical surveys through estimating the distances to galaxies (redshifts) from photometry. Due to the massive scale of data coming from these new and upcoming sky surveys, machine learning techniques using galaxy photometry are increasingly adopted to predict galactic redshifts which are important for inferring cosmological parameters such as the nature of dark energy. Associated uncertainty estimates are also critical measurements, however, common machine learning methods typically provide only point estimates and lack uncertainty information as outputs. We turn to Bayesian neural networks (BNNs) as a promising way to provide accurate predictions of redshift values. We have compiled a new galaxy training dataset from the Hyper Suprime-Cam Survey, designed to mimic large surveys, but over a smaller portion of the sky. We evaluate the performance and accuracy of photometric redshift (photo-z) predictions from photometry using machine learning, astronomical and probabilistic metrics. We find that while the Bayesian neural network did not perform as well as non-Bayesian neural networks if evaluated solely by point estimate photo-z values, BNNs can provide uncertainty estimates that are necessary for cosmology.

We present the largest low frequency (120~MHz) arcminute resolution image of the radio synchrotron background (RSB) to date, and its corresponding angular power spectrum of anisotropies (APS) with angular scales ranging from $3^\circ$ to $0.3^\prime$. We show that the RSB around the North Celestial Pole has a significant excess anisotropy power at all scales over a model of unclustered point sources based on source counts of known source classes. This anisotropy excess, which does not seem attributable to the diffuse Galactic emission, could be linked to the surface brightness excess of the RSB. To better understand the information contained within the measured APS, we model the RSB varying the brightness distribution, size, and angular clustering of potential sources. We show that the observed APS could be produced by a population of faint clustered point sources only if the clustering is extreme and the size of the Gaussian clusters is $\lesssim 1'$. We also show that the observed APS could be produced by a population of faint diffuse sources with sizes $\lesssim 1'$, and this is supported by features present in our image. Both of these cases would also cause an associated surface brightness excess. These classes of sources are in a parameter space not well probed by even the deepest radio surveys to date.

We summarize the second radio synchrotron background workshop, which took place on 2022 June 15–17 in Barolo, Italy. This meeting was convened because available measurements of the diffuse radio zero level continue to suggest that it is several times higher than can be attributed to known Galactic and extragalactic sources and processes, rendering it the least well-understood electromagnetic background at present and a major outstanding question in astrophysics. The workshop agreed on the next priorities for investigations of this phenomenon, which include searching for evidence of the radio Sunyaev–Zel’dovich effect, carrying out cross-correlation analyses of radio emission with other tracers, and supporting the completion of the 310 MHz absolutely calibrated sky map project.

We present the first targeted measurement of the power spectrum of anisotropies of the radio synchrotron background, at 140 MHz where it is the overwhelmingly dominant photon background. This measurement is important for understanding the background level of radio sky brightness, which is dominated by steep-spectrum synchrotron radiation at frequencies below ν ∼ 0.5 GHz and has been measured to be significantly higher than that which can be produced by known classes of extragalactic sources and most models of Galactic halo emission. We determine the anisotropy power spectrum on scales ranging from 2○ to 0.2′ with LOFAR observations of two 18 deg2 fields — one centered on the Northern hemisphere coldest patch of radio sky where the Galactic contribution is smallest and one offset from that location by 15○. We find that the anisotropy power is higher than that attributable to the distribution of point sources above 100 μJy in flux. This level of radio anisotropy power indicates that if it results from point sources, those sources are likely at low fluxes and incredibly numerous, and likely clustered in a specific manner.

We explore the evolution of the X-ray luminosity function of quasars and the intrinsic correlation between the X-ray and 2500 angstrom ultraviolet luminosities, utilizing techniques verified in previous works and a sample of over 4000 quasars detected with Chandra and XMM-Newton in the range 0 < z < 5. We find that quasars have undergone significantly less evolution with redshift in their total X-ray luminosity than in other wave bands. We then determine that the best-fit intrinsic power-law correlation between the X-ray and ultraviolet luminosities, of the form L'(X) proportional to (L'(UV))(gamma), is gamma = 0.28 +/- 0.03, and we derive the luminosity function and density evolution in the X-ray band. We discuss the implications of these results for models of quasar systems.

We present results of using a basic binary classification neural network model to identify likely catastrophic outlier photometric redshift estimates of individual galaxies, based only on the galaxies’ measured photometric band magnitude values. We find that a simple implementation of this classification can identify a significant fraction of galaxies with catastrophic outlier photometric redshift estimates while falsely categorizing only a much smaller fraction of non-outliers. These methods have the potential to reduce the errors introduced into science analyses by catastrophic outlier photometric redshift estimates.

We explore whether an independent determination of the distance-redshift relation, and hence cosmological model parameters, can be obtained from the apparent correlations between two different waveband luminosities or fluxes, as has been claimed in recent works using the X-ray and ultraviolet luminosities and fluxes of quasars. We show that such an independent determination is possible only if the correlation between luminosities is obtained independent of the cosmological model, and measured fluxes and redshifts; for example, being based on sound theoretical models or unrelated observations. In particular, we show that if the correlation is determined empirically from two luminosities obtained from fluxes and redshifts, then the method suffers from circularity. In case, as claimed in recent works, the observed correlation between fluxes in very narrow redshift bins are used as proxy for the luminosity correlation, then we show that one is dealing with a pure tautology with no information on distances and cosmological model. We argue that the problem arises because of the incomplete treatment of the correlation and we use numerical methods with a joint X-ray and ultraviolet quasar data set to demonstrate this shortcoming.

We present results of using individual galaxies’ probability distribution over redshift as a method of identifying potential catastrophic outliers in empirical photometric redshift estimation. In the course of developing this approach we develop a method of modification of the redshift distribution of training sets to improve both the baseline accuracy of high redshift ( z > 1.5) estimation as well as catastrophic outlier mitigation. We demonstrate these using two real test data sets and one simulated test data set spanning a wide redshift range (0 < z < 4). Results presented here inform an example “prescription” that can be applied as a realistic photometric redshift estimation scenario for a hypothetical large-scale survey. We find that with appropriate optimization, we can identify a significant percentage (>30%) of catastrophic outlier galaxies while simultaneously incorrectly flagging only a small percentage (<7% and in many cases <3%) of non-outlier galaxies as catastrophic outliers. We find also that our training set redshift distribution modification results in a significant (>10) percentage point decrease of outlier galaxies for z > 1.5 with only a small (<3) percentage point increase of outlier galaxies for z < 1.5 compared to the unmodified training set. In addition, we find that this modification can in some cases cause a significant (∼20) percentage point decrease of galaxies which are non-outliers but which have been incorrectly identified as outliers, while in other cases cause only a small (<1) increase in this metric.

It has recently become apparent that the background level of diffuse radio emission on the sky is significantly higher than the level that can result from known extragalactic radio source classes or our Galaxy given our current understanding of its large-scale structure.~ In contrast to the more well-known and well-constrained cosmological and astrophysical backgrounds at microwave, infrared, optical/UV, X-ray, and gamma-ray wavelengths, this ``radio synchrotron background'' at radio wavelengths provides clear motivation for considering the possibilities of new astrophysical sources and new particle-based emission mechanisms.

We present results of using individual galaxies' redshift probability information derived from a photometric redshift (photo-z) algorithm, SPIDERz, to identify potential catastrophic outliers in photometric redshift determinations. By using two test data sets comprised of COSMOS multi-band photometry spanning a wide redshift range (050%) of the catastrophic outlier galaxies, while only flagging a small fraction (<5%) of the total non-outlier galaxies, depending on parameter choices. The fraction of non-outlier galaxies flagged varies significantly with redshift and magnitude, however. We examine the performance of this strategy in photo-z determinations using a range of flagging parameter values. These results could potentially be useful for utilization of photometric redshifts in future large scale surveys where catastrophic outliers are particularly detrimental to the science goals.

We explore the general question of correlations among different waveband luminosities in a flux-limited multiband observational data set. Such correlations, often observed for astronomical sources, may either be intrinsic or induced by the redshift evolution of the luminosities and the data truncation due to the flux limits. We first address this question analytically. We then use simulated flux-limited data with three different known intrinsic luminosity correlations, and prescribed luminosity functions and evolution similar to the ones expected for quasars. We explore how the intrinsic nature of luminosity correlations can be deduced, including exploring the efficacy of partial correlation analysis with redshift binning in determining whether luminosity correlations are intrinsic and finding the form of the intrinsic correlation. By applying methods that we have developed in recent works, we show that we can recover the true cosmological evolution of the luminosity functions and the intrinsic correlations between the luminosities. Finally, we demonstrate the methods for determining intrinsic luminosity correlations on actual observed samples of quasars with mid-infrared, radio, and optical fluxes and redshifts, finding that the luminosity-luminosity correlation is significantly stronger between mid-infrared and optical than that between radio and optical luminosities, supporting the canonical jet-launching and heating model of active galaxies.

We summarize the radio synchrotron background workshop that took place 2017 July 19–21 at the University of Richmond. This first scientific meeting dedicated to the topic was convened because current measurements of the diffuse radio monopole reveal a surface brightness that is several times higher than can be straightforwardly explained by known Galactic and extragalactic sources and processes, rendering it by far the least well understood photon background at present. It was the conclusion of a majority of the participants that the radio monopole level is at or near that reported by the ARCADE 2 experiment and inferred from several absolutely calibrated zero-level lower frequency radio measurements, and unanimously agreed that the production of this level of surface brightness, if confirmed, represents a major outstanding question in astrophysics. The workshop reached a consensus on the next priorities for investigations of the radio synchrotron background.

Aims: We present a custom support vector machine classification package for photometric redshift estimation, including comparisons with other methods. We also explore the efficacy of including galaxy shape information in redshift estimation. Support vector machines, a type of machine learning, utilize optimization theory and supervised learning algorithms to construct predictive models based on the information content of data in a way that can treat different input features symmetrically. Methods: The custom support vector machine package we have developed is designated SPIDERz and made available to the community. As test data for evaluating performance and comparison with other methods, we apply SPIDERz to four distinct data sets: 1) the publicly available portion of the PHAT-1 catalog based on the GOODS-N field with spectroscopic redshifts in the range $z < 3.6$, 2) 14365 galaxies from the COSMOS bright survey with photometric band magnitudes, morphology, and spectroscopic redshifts inside $z < 1.4$, 3) 3048 galaxies from the overlap of COSMOS photometry and morphology with 3D-HST spectroscopy extending to $z < 3.9$, and 4) 2612 galaxies with five-band photometric magnitudes and morphology from the All-wavelength Extended Groth Strip International Survey and $z < 1.57$. Results: We find that SPIDER-z achieves results competitive with other empirical packages on the PHAT-1 data, and performs quite well in estimating redshifts with the COSMOS and AEGIS data, including in the cases of a large redshift range ($0 < z < 3.9$). We also determine from analyses with both the COSMOS and AEGIS data that the inclusion of morphological information does not have a statistically significant benefit for photometric redshift estimation with the techniques employed here.

We use surface brightness contour maps of nearby edge-on spiral galaxies to determine whether extended bright radio halos are common. In particular, we test a recent model of the spatial structure of the diffuse radio continuum by Subrahmanyan & Cowsik which posits that a substantial fraction of the observed high-latitude surface brightness originates from an extended Galactic halo of uniform emissivity. Measurements of the axial ratio of emission contours within a sample of normal spiral galaxies at 1500 MHz and below show no evidence for such a bright, extended radio halo. Either the Galaxy is atypical compared to nearby quiescent spirals or the bulk of the observed high-latitude emission does not originate from this type of extended halo.