The precise characterization and mitigation of systematic effects is one of the biggest roadblocks impeding the detection of the fluctuations of cosmological 21 cm signals. Missing data in radio cosmological experiments, often due to radio frequency interference (RFI), pose a particular challenge to power spectrum analysis as this could lead to the ringing of bright foreground modes in the Fourier space, heavily contaminating the cosmological signals. Here we show that the problem of missing data becomes even more arduous in the presence of systematic effects. Using a realistic numerical simulation, we demonstrate that partially flagged data combined with systematic effects can introduce significant foreground ringing. We show that such an effect can be mitigated through inpainting the missing data. We present a rigorous statistical framework that incorporates the process of inpainting missing data into a quadratic estimator of the 21 cm power spectrum. Under this framework, the uncertainties associated with our inpainting method and its impact on power spectrum statistics can be understood. These results are applied to the latest Phase II observations taken by the Hydrogen Epoch of Reionization Array, forming a crucial component in power spectrum analyses as we move toward detecting 21 cm signals in the ever more noisy RFI environment.

Interferometric experiments designed to detect the highly redshifted 21-cm signal from neutral hydrogen are producing increasingly stringent constraints on the 21-cm power spectrum, but some k-modes remain systematics-dominated. Mutual coupling is a major systematic that must be overcome in order to detect the 21-cm signal, and simulations that reproduce effects seen in the data can guide strategies for mitigating mutual coupling. In this paper, we analyse 12 nights of data from the Hydrogen Epoch of Reionization Array and compare the data against simulations that include a computationally efficient and physically motivated semi-analytic treatment of mutual coupling. We find that simulated coupling features qualitatively agree with coupling features in the data; however, coupling features in the data are brighter than the simulated features, indicating the presence of additional coupling mechanisms not captured by our model. We explore the use of fringe-rate filters as mutual coupling mitigation tools and use our simulations to investigate the effects of mutual coupling on a simulated cosmological 21-cm power spectrum in a "worst case" scenario where the foregrounds are particularly bright. We find that mutual coupling contaminates a large portion of the "EoR Window", and the contamination is several orders-of-magnitude larger than our simulated cosmic signal across a wide range of cosmological Fourier modes. While our fiducial fringe-rate filtering strategy reduces mutual coupling by roughly a factor of 100 in power, a non-negligible amount of coupling cannot be excised with fringe-rate filters, so more sophisticated mitigation strategies are required.

Detection of the faint 21 cm line emission from the Cosmic Dawn and Epoch of Reionisation will require not only exquisite control over instrumental calibration and systematics to achieve the necessary dynamic range of observations but also validation of analysis techniques to demonstrate their statistical properties and signal loss characteristics. A key ingredient in achieving this is the ability to perform high-fidelity simulations of the kinds of data that are produced by the large, many-element, radio interferometric arrays that have been purpose-built for these studies. The large scale of these arrays presents a computational challenge, as one must simulate a detailed sky and instrumental model across many hundreds of frequency channels, thousands of time samples, and tens of thousands of baselines for arrays with hundreds of antennas. In this paper, we present a fast matrix-based method for simulating radio interferometric measurements (visibilities) at the necessary scale. We achieve this through judicious use of primary beam interpolation, fast approximations for coordinate transforms, and a vectorised outer product to expand per-antenna quantities to per-baseline visibilities, coupled with standard parallelisation techniques. We validate the results of this method, implemented in the publicly-available matvis code, against a high-precision reference simulator, and explore its computational scaling on a variety of problems.

Observations with radio arrays that target the 21-cm signal originating from the early Universe suffer from a variety of systematic effects. An important class of these are reflections and spurious couplings between antennas. We apply a Hamiltonian Monte Carlo sampler to the modelling and mitigation of these systematics in simulated Hydrogen Epoch of Reionisation Array (HERA) data. This method allows us to form statistical uncertainty estimates for both our models and the recovered visibilities, which is an important ingredient in establishing robust upper limits on the Epoch of Reionisation (EoR) power spectrum. In cases where the noise is large compared to the EoR signal, this approach can constrain the systematics well enough to mitigate them down to the noise level for both systematics studied. Where the noise is smaller than the EoR, our modelling can mitigate the majority of the reflections with there being only a minor level of residual systematics, while cross-coupling sees essentially complete mitigation. Our approach performs similarly to existing filtering/fitting techniques used in the HERA pipeline, but with the added benefit of rigorously propagating uncertainties. In all cases it does not significantly attenuate the underlying signal.

Radio interferometers targeting the 21cm brightness temperature fluctuations at high redshift are subject to systematic effects that operate over a range of different time-scales. These can be isolated by designing appropriate Fourier filters that operate in fringe-rate (FR) space, the Fourier pair of local sidereal time. Applications of FR filtering include separating effects that are correlated with the rotating sky versus those relative to the ground, down-weighting emission in the primary beam sidelobes, and suppressing noise. FR filtering causes the noise contributions to the visibility data to become correlated in time, however, making interpretation of subsequent averaging and error estimation steps more subtle. In this paper, we describe fringe-rate filters that are implemented using discrete prolate spheroidal sequences, and designed for two different purposes-beam sidelobe/horizon suppression (the 'mainlobe' filter), and ground-locked systematics removal (the 'notch' filter). We apply these to simulated data, and study how their properties affect visibilities and power spectra generated from the simulations. Included is an introduction to fringe-rate filtering and a demonstration of fringe-rate filters applied to simple situations to aid understanding.

The 21 cm transition from neutral Hydrogen promises to be the best observational probe of the Epoch of Reionization (EoR). This has led to the construction of low-frequency radio interferometric arrays, such as the Hydrogen Epoch of Reionization Array (HERA), aimed at systematically mapping this emission for the first time. Precision calibration, however, is a requirement in 21 cm radio observations. Due to the spatial compactness of HERA, the array is prone to the effects of mutual coupling, which inevitably lead to non-smooth calibration errors that contaminate the data. When unsmooth gains are used in calibration, intrinsically spectrally smooth foreground emission begins to contaminate the data in a way that can prohibit a clean detection of the cosmological EoR signal. In this paper, we show that the effects of mutual coupling on calibration quality can be reduced by applying custom time-domain filters to the data prior to calibration. We find that more robust calibration solutions are derived when filtering in this way, which reduces the observed foreground power leakage. Specifically, we find a reduction of foreground power leakage by 2 orders of magnitude at k|| approximate to 0 . 5 h Mpc(-1).

The key to detecting neutral hydrogen during the epoch of reionization (EoR) is to separate the cosmological signal from the dominating foreground radiation. We developed direct optimal mapping (DOM) to map interferometric visibilities; it contains only linear operations, with full knowledge of point spread functions from visibilities to images. Here, we demonstrate a fast Fourier transform-based image power spectrum and its window functions computed from the DOM images. We use noiseless simulation, based on the Hydrogen Epoch of Reionization Array Phase I configuration, to study the image power spectrum properties. The window functions show <10(-11) of the integrated power leaks from the foreground-dominated region into the EoR window; the 2D and 1D power spectra also verify the separation between the foregrounds and the EoR.

This paper presents the design and deployment of the Hydrogen Epoch of Reionization Array (HERA) phase II system. HERA is designed as a staged experiment targeting 21 cm emission measurements of the Epoch of Reionization. First results from the phase I array are published as of early 2022, and deployment of the phase II system is nearing completion. We describe the design of the phase II system and discuss progress on commissioning and future upgrades. As HERA is a designated Square Kilometer Array (SKA) pathfinder instrument, we also show a number of "case studies" that investigate systematics seen while commissioning the phase II system, which may be of use in the design and operation of future arrays. Common pathologies are likely to manifest in similar ways across instruments, and many of these sources of contamination can be mitigated once the source is identified.

We report the most sensitive upper limits to date on the 21 cm epoch of reionization power spectrum using 94 nights of observing with Phase I of the Hydrogen Epoch of Reionization Array (HERA). Using similar analysis techniques as in previously reported limits, we find at 95% confidence that Δ 2 ( k = 0.34 h Mpc −1 ) ≤ 457 mK 2 at z = 7.9 and that Δ 2 ( k = 0.36 h Mpc −1 ) ≤ 3496 mK 2 at z = 10.4, an improvement by a factor of 2.1 and 2.6, respectively. These limits are mostly consistent with thermal noise over a wide range of k after our data quality cuts, despite performing a relatively conservative analysis designed to minimize signal loss. Our results are validated with both statistical tests on the data and end-to-end pipeline simulations. We also report updated constraints on the astrophysics of reionization and the cosmic dawn. Using multiple independent modeling and inference techniques previously employed by HERA Collaboration, we find that the intergalactic medium must have been heated above the adiabatic cooling limit at least as early as z = 10.4, ruling out a broad set of so-called “cold reionization” scenarios. If this heating is due to high-mass X-ray binaries during the cosmic dawn, as is generally believed, our result’s 99% credible interval excludes the local relationship between soft X-ray luminosity and star formation and thus requires heating driven by evolved low-metallicity stars.

The Square Kilometre Array (SKA) telescope is designed to operate concurrently over multiple frequency bands across two continents. The low frequency instrument which will be deployed in Western Australia will operate in the 50 MHz to 350 MHz band using thousands of broadband log-period antennas (SKALA type). In this paper, we present a verification array consisting of 64 SKALA4 antennas operating at Lord’s Bridge, Cambridge. The array utilises much of the technology proposed for SKA1-LOW, including RF-over-Fibre WDM optical transmitters. Some changes to the system architecture have been made, including switchable filters to allow operation in a higher RFI environment, especially from FM and digital radio. In this paper, we present an overview of the system which will be used for validation of EM models proposed for SKA1-LOW.

ABSTRACT Radio interferometers aiming to measure the power spectrum of the redshifted 21 cm line during the Epoch of Reionization (EoR) need to achieve an unprecedented dynamic range to separate the weak signal from overwhelming foreground emissions. Calibration inaccuracies can compromise the sensitivity of these measurements to the effect that a detection of the EoR is precluded. An alternative to standard analysis techniques makes use of the closure phase, which allows one to bypass antenna-based direction-independent calibration. Similarly to standard approaches, we use a delay spectrum technique to search for the EoR signal. Using 94 nights of data observed with Phase I of the Hydrogen Epoch of Reionization Array (HERA), we place approximate constraints on the 21 cm power spectrum at z = 7.7. We find at 95 per cent confidence that the 21 cm EoR brightness temperature is ≤(372)2 ‘pseudo’ mK2 at 1.14 ‘pseudo’ h Mpc−1, where the ‘pseudo’ emphasizes that these limits are to be interpreted as approximations to the actual distance scales and brightness temperatures. Using a fiducial EoR model, we demonstrate the feasibility of detecting the EoR with the full array. Compared to standard methods, the closure phase processing is relatively simple, thereby providing an important independent check on results derived using visibility intensities, or related.

Radio Frequency Interference (RFI) is one of the systematic challenges preventing 21cm interferometric instruments from detecting the Epoch of Reionization. To mitigate the effects of RFI on data analysis pipelines, numerous inpaint techniques have been developed to restore RFI corrupted data. We examine the qualitative and quantitative errors introduced into the visibilities and power spectrum due to inpainting. We perform our analysis on simulated data as well as real data from the Hydrogen Epoch of Reionization Array (HERA) Phase 1 upper limits. We also introduce a convolutional neural network that capable of inpainting RFI corrupted data in interferometric instruments. We train our network on simulated data and show that our network is capable at inpainting real data without requiring to be retrained. We find that techniques that incorporate high wavenumbers in delay space in their modeling are best suited for inpainting over narrowband RFI. We also show that with our fiducial parameters Discrete Prolate Spheroidal Sequences (DPSS) and CLEAN provide the best performance for intermittent “narrowband” RFI while Gaussian Progress Regression (GPR) and Least Squares Spectral Analysis (LSSA) provide the best performance for larger RFI gaps. However we caution that these qualitative conclusions are sensitive to the chosen hyperparameters of each inpainting technique. We find these results to be consistent in both simulated and real visibilities. We show that all inpainting techniques reliably reproduce foreground dominated modes in the power spectrum. Since the inpainting techniques should not be capable of reproducing noise realizations, we find that the largest errors occur in the noise dominated delay modes. We show that in the future, as the noise level of the data comes down, CLEAN and DPSS are most capable of reproducing the fine frequency structure in the visibilities of HERA data.

Combining the visibilities measured by an interferometer to form a cosmological power spectrum is a complicated process. In a delay-based analysis, the mapping between instrumental and cosmological space is not a one-to-one relation. Instead, neighbouring modes contribute to the power measured at one point, with their respective contributions encoded in the window functions. To better understand the power measured by an interferometer, we assess the impact of instrument characteristics and analysis choices on these window functions. Focusing on the Hydrogen Epoch of Reionization Array (HERA) as a case study, we find that long-baseline observations correspond to enhanced low-k tails of the window functions, which facilitate foreground leakage, whilst an informed choice of bandwidth and frequency taper can reduce said tails. With simple test cases and realistic simulations, we show that, apart from tracing mode mixing, the window functions help accurately reconstruct the power spectrum estimator of simulated visibilities. The window functions depend strongly on the beam chromaticity, and less on its spatial structure - a Gaussian approximation, ignoring side lobes, is sufficient. Finally, we investigate the potential of asymmetric window functions, down-weighting the contribution of low-k power to avoid foreground leakage. The window functions presented here correspond to the latest HERA upper limits for the full Phase I data. They allow an accurate reconstruction of the power spectrum measured by the instrument and will be used in future analyses to confront theoretical models and data directly in cylindrical space.

We present a Bayesian jackknife test for assessing the probability that a data set contains biased subsets, and, if so, which of the subsets are likely to be biased. The test can be used to assess the presence and likely source of statistical tension between different measurements of the same quantities in an automated manner. Under certain broadly applicable assumptions, the test is analytically tractable. We also provide an open source code, CHIBORG, that performs both analytic and numerical computations of the test on general Gaussian-distributed data. After exploring the information theoretical aspects of the test and its performance with an array of simulations, we apply it to data from the Hydrogen Epoch of Reionization Array (HERA) to assess whether different sub-seasons of observing can justifiably be combined to produce a deeper 21cm power spectrum upper limit. We find that, with a handful of exceptions, the HERA data in question are statistically consistent and this decision is justified. We conclude by pointing out the wide applicability of this test, including to CMB experiments and the $H_0$ tension.

Recently, the Hydrogen Epoch of Reionization Array (HERA) has produced the experiment’s first upper limits on the power spectrum of 21 cm fluctuations at z ∼ 8 and 10. Here, we use several independent theoretical models to infer constraints on the intergalactic medium (IGM) and galaxies during the epoch of reionization from these limits. We find that the IGM must have been heated above the adiabatic-cooling threshold by z ∼ 8, independent of uncertainties about IGM ionization and the radio background. Combining HERA limits with complementary observations constrains the spin temperature of the z ∼ 8 neutral IGM to 27 K 〈T¯S〉 630 K (2.3 K 〈T¯S〉 640 K) at 68% (95%) confidence. They therefore also place a lower bound on X-ray heating, a previously unconstrained aspects of early galaxies. For example, if the cosmic microwave background dominates the z ∼ 8 radio background, the new HERA limits imply that the first galaxies produced X-rays more efficiently than local ones. The z ∼ 10 limits require even earlier heating if dark-matter interactions cool the hydrogen gas. If an extra radio background is produced by galaxies, we rule out (at 95% confidence) the combination of high radio and low X-ray luminosities of L r,ν /SFR > 4 × 1024 W Hz−1 M⊙−1 yr and L X /SFR < 7.6 × 1039 erg s−1 M⊙−1 yr. The new HERA upper limits neither support nor disfavor a cosmological interpretation of the recent Experiment to Detect the Global EOR Signature (EDGES) measurement. The framework described here provides a foundation for the interpretation of future HERA results.

We present a framework for identifying and flagging malfunctioning antennas in large radio interferometers. We outline two distinct categories of metrics designed to detect outliers along known failure modes of large arrays: cross-correlation metrics, based on all antenna pairs, and auto-correlation metrics, based solely on individual antennas. We define and motivate the statistical framework for all metrics used, and present tailored visualizations that aid us in clearly identifying new and existing systematics. We implement these techniques using data from 105 antennas in the Hydrogen Epoch of Reionization Array (HERA) as a case study. Finally, we provide a detailed algorithm for implementing these metrics as flagging tools on real data sets.

Motivated by the desire for wide-field images with well-defined statistical properties for 21 cm cosmology, we implement an optimal mapping pipeline that computes a maximum likelihood estimator for the sky using the interferometric measurement equation. We demonstrate this “direct optimal mapping” with data from the Hydrogen Epoch of Reionization (HERA) Phase I observations. After validating the pipeline with simulated data, we develop a maximum likelihood figure-of-merit for comparing four sky models at 166 MHz with a bandwidth of 100 kHz. The HERA data agree with the GLEAM catalogs to < 10%. After subtracting the GLEAM point sources, the HERA data discriminate between the different continuum sky models, providing most support for the model of Byrne et al. We report the computation cost for mapping the HERA Phase I data and project the computation for the HERA 320-antenna data; both are feasible with a modern server. The algorithm is broadly applicable to other interferometers and is valid for wide-field and noncoplanar arrays.

We describe the validation of the HERA Phase I software pipeline by a series of modular tests, building up to an end-to-end simulation. The philosophy of this approach is to validate the software and algorithms used in the Phase I upper-limit analysis on wholly synthetic data satisfying the assumptions of that analysis, not addressing whether the actual data meet these assumptions. We discuss the organization of this validation approach, the specific modular tests performed, and the construction of the end-to-end simulations. We explicitly discuss the limitations in scope of the current simulation effort. With mock visibility data generated from a known analytic power spectrum and a wide range of realistic instrumental effects and foregrounds, we demonstrate that the current pipeline produces power spectrum estimates that are consistent with known analytic inputs to within thermal noise levels (at the 2σ level) for k > 0.2h Mpc−1 for both bands and fields considered. Our input spectrum is intentionally amplified to enable a strong “detection” at k ∼ 0.2 h Mpc−1—at the level of ∼25σ—with foregrounds dominating on larger scales and thermal noise dominating at smaller scales. Our pipeline is able to detect this amplified input signal after suppressing foregrounds with a dynamic range (foreground to noise ratio) of ≳107. Our validation test suite uncovered several sources of scale-independent signal loss throughout the pipeline, whose amplitude is well-characterized and accounted for in the final estimates. We conclude with a discussion of the steps required for the next round of data analysis.

We report upper limits on the Epoch of Reionization 21 cm power spectrum at redshifts 7.9 and 10.4 with 18 nights of data (∼36 hr of integration) from Phase I of the Hydrogen Epoch of Reionization Array (HERA). The Phase I data show evidence for systematics that can be largely suppressed with systematic models down to a dynamic range of ∼10 9 with respect to the peak foreground power. This yields a 95% confidence upper limit on the 21 cm power spectrum of Δ 21 2 ≤ ( 30.76 ) 2 mK 2 at k = 0.192 h Mpc −1 at z = 7.9, and also Δ 21 2 ≤ ( 95.74 ) 2 mK 2 at k = 0.256 h Mpc −1 at z = 10.4. At z = 7.9, these limits are the most sensitive to date by over an order of magnitude. While we find evidence for residual systematics at low line-of-sight Fourier k ∥ modes, at high k ∥ modes we find our data to be largely consistent with thermal noise, an indicator that the system could benefit from deeper integrations. The observed systematics could be due to radio frequency interference, cable subreflections, or residual instrumental cross-coupling, and warrant further study. This analysis emphasizes algorithms that have minimal inherent signal loss, although we do perform a careful accounting in a companion paper of the small forms of loss or bias associated with the pipeline. Overall, these results are a promising first step in the development of a tuned, instrument-specific analysis pipeline for HERA, particularly as Phase II construction is completed en route to reaching the full sensitivity of the experiment.