This chapter summarizes the observational lessons learned after two decades of observations of the Cosmic Dawn (CD) and Epoch of Reionization (EoR) with SKAO pathfinders and precursors. We will describe the effort towards building accurate simulation pipelines for actual observations and summarize the approaches that different groups have taken to calibrate and mitigate systematic effects such as sky model incompleteness, limited instrument models and antenna mutual coupling. We conclude by discussing the impact that these lessons may have on the design and analysis of upcoming SKAO observations of the Cosmic Dawn and Epoch of Reionization.

The embedded element patterns (EEPs) of wideband antenna arrays can exhibit sharp spectral variations caused by mutual coupling between antennas. This, in turn, increases the time-delay spread of the impulse response, a critical performance metric in modern radio astronomy. To minimise this chromaticity, we propose an optimisation of the array layout using aperiodic tiling, specifically the hat polykite pattern. Two in-house methods are combined: a fast electromagnetic solver for rapid generation of EEPs for a given layout, and an optimisation scheme based on PolyChord's nested-sampling algorithm to efficiently explore the antenna position parameter space and determine the optimal configuration. Results show that aperiodic tiling yields EEPs with improved spectral smoothness compared to existing layouts.

Mutual coupling between antennas has emerged as the dominant direction-dependent corruption in dense aperture arrays, imprinting pronounced sub-MHz spatial and spectral structure that compromises the time-gating and foreground-separation strategies used to isolate the faint 21-cm signal. In this work, we introduce Direct Primary Beam Correction, a domain-agnostic framework for reconstructing the far-field radiation pattern relative to an arbitrary reference via a regularised, direction-weighted linear inversion of stacked Jones matrices, thereby enabling the removal of direction-dependent distortions such as mutual coupling. Using full-wave electromagnetic simulations of SKA-Low, we demonstrate that this framework reconstructs the radiation pattern down to the numerical noise floor within a suitably conditioned field of view, with the reconstruction accuracy governed by the regularised inversion and the fidelity of the underlying beam model. Applying the framework to a simulated 4-hour observation of the EoR0 field in the 122–134 MHz band, we identify two principal implications for 21-cm power-spectrum analysis. First, restricting the correction to the main lobe and near sidelobes is inadequate: chromatic grating lobe contributions, whether left insufficiently or entirely uncorrected, continue to contaminate the EoR window. Second, mutual-coupling-induced contamination is temporally coherent and, being anchored to the fixed array geometry, does not average down across snapshots as the EoR field is tracked. Direct primary beam correction, therefore, provides a computationally efficient means of mitigating mutual coupling; however, robust recovery of the EoR window necessitates either full-sky correction or explicit separation of main-beam and sidelobe contributions prior to power-spectrum estimation.

Radio telescopes composed of phased arrays, called stations, produce visibilities by cross-correlating signals of beamformed antenna voltages from each station. For large telescopes with many stations, accurately modelling visibilities presents computational challenges due to the need for full-wave simulations of each antenna's electromagnetic response, including Mutual Coupling (MC) effects, and the evaluation of array patterns across many directions and frequencies. This paper leverages two in-house tools: the Fast Array Simulation Tool for rapid electromagnetic simulations and the OSKAR simulator for parallelized interferometric computations. For the first time, we analyze the impact of intra-station MC on the visibility time-delay impulse response across different station layouts, focusing on the 224 core stations of the Square Kilometre Array Low telescope. We show that MC significantly broadens the delay spread of instrument response, creating a key limitation for 21-cm science experiments while offering new insights into challenges posed by MC in large-scale radio telescopes.

For the successful calibration and operation of the next generation of radio-instruments we require ever more detailed understanding of the antenna systems. We present a method to analyze the broadband frequency evolution of current modes within radio-frequency antennas, including the effects of mutual coupling when such antennas are operated in a subwavelength spaced array. We demonstrate this method using the SKA-low log periodic dipole antenna, observing the smooth evolution of current modes in the isolated antenna between 50 and 150 MHz. Further, we identify multiple narrowband resonancelike features within the current modes when the antenna is exposed to mutual coupling within an array. These narrow features are seen within both the excited antenna and passive antennas when operated within a 256 element station. Further, we note that there is significant current activity observed on the vertical boom and the perpendicular polarization on the passive antennas.

We investigate the impact of mutual coupling (MC) between antennas on the time-delay power spectrum response of the core of the SKA-Low radio telescope. Using two in-house tools - Fast Array Simulation Tool (FAST; a fast full-wave electromagnetic solver) and Oxford Square Kilometre Array (OSKAR) (a GPU-accelerated radio telescope simulator) - we simulate station beams and compute visibilities for various station layouts (regular, sunflower, and random). Simulations are conducted in an epoch of reionization subband between 120-150 MHz, with a fine spectral resolution of 100 kHz, enabling the investigation of longer delays. Our results show that MC effects significantly increase foreground leakage into longer delays, especially for regular station layouts. For 21-cm science, foreground spill over into the 21-cm window extends beyond k parallel to similar to 2h(-1) Mpc for all station layouts and across all k(perpendicular to) modes, completely obscuring the detection window. We find that attempting to remove the foreground contribution from the visibilities using an approximated beam model - either based on the average embedded element pattern or by interpolating the embedded element patterns from a coarse channel rate of 781 kHz - results in residuals around similar to 10(11)mK(2)h(-3)Mpc(3). This is still seven orders of magnitude brighter than the expected level of the EoR signal (similar to 10(4) mK(2)h(-3)Mpc(3)). We also find that station beam models with at least 4-5 significant digits in the far-field pattern and high spectral resolution are needed for effective foreground removal. Our research provides critical insights into the role of MC in SKA-Low experiments and highlights the computational challenges of fully integrating array patterns that account for MC effects into processing pipelines.

Mutual coupling (MC) among SKA-Low's log-periodic antennas exhibits narrowband resonances, which introduce sharp spectral variations in the embedded element patterns (EEP). These effects compromise time-gating techniques that rely on smooth beam responses to separate the faint 21-cm cosmological signal from bright astrophysical foregrounds. In this work, we analyse the time-delay power spectrum of the SKA-Low core across 60-150 MHz within the Epoch of Reionization (EoR) band using a dedicated simulation pipeline. Our results demonstrate that MC significantly increases spectral complexity, leading to foreground leakage throughout the 21-cm detection window. This contamination challenges current foreground mitigation strategies, demonstrating the need for high-fidelity beam modelling within 21-cm experiments.

ABSTRACT The Cosmic Dawn was marked by the formation of the first stars, and preceded the Epoch of Reionization (EoR), when the Universe underwent a fundamental transformation caused by the radiation from these first stars and galaxies. Interferometric 21-cm experiments aim to probe redshifted neutral hydrogen signals from these periods, constraining the conditions of the early Universe. The SKA-Low instrument of the Square Kilometre Array (SKA) is envisaged to be the largest and most sensitive radio telescope at metre and centimetre wavelengths. The latest Aperture Array Verification Systems feature 7-m coaxial transmission lines connecting the low noise amplifiers to optical transmitters at the front of the analogue-receiving chain. An impedance mismatch between these components results in a partially reflected electromagnetic signal, introducing chromatic aberrations into the instrument bandpass. This causes power from the foreground signals to appear at higher delays, potentially contaminating the ‘EoR window’, a region in which the 21-cm signal should be detectable. We present an end-to-end simulation pipeline for SKA-Low using a composite sky model combining radio foregrounds from the Galactic and Extragalactic All-Sky MWA (GLEAM) Survey, Haslam 408 MHz, and a 1.5-cGpc 21-cm brightness temperature cube generated with the 21cm space simulator. We derive a model for the scattering parameters of a coaxial transmission line in terms of its specifications and bulk material properties. Assuming identical cables of length $\le 15.0$ m with impedance mismatch $\le 10\, \Omega$, the reflection is confined below the EoR window. However, we demonstrate that even a 0.1 per cent length tolerance introduces contamination with an absolute fractional difference of ~10 per cent across all accessible k-modes.

The most recent Aperture Array Verification System for the SKA-Low, AAVS3, has been designed with a regular sunflower head ‘Vogel’ layout with a 19 m radius. We use a fast Method-of-Moments solver to investigate the effects of mutual coupling of this style of regular layout in comparison to a more randomised layout. We show that this layout produces a blind spot at zenith at 125 MHz, where the embedded element pattern and station beam show a drop of around 5 dB. We then demonstrate that randomising the locations of the elements reduces the scale of this drop in beam, with greater randomisation producing greater reduction. We find that randomising each element position within an area of around 1.5 m radius is sufficient to eliminate this blind spot.

The SKA-low observatory aims to use 512 stations of 256 antenna phased array to perform the next level of radio frequency astronomical observations. This most recent Aperture Array Verification System, AAVS3, operates a regular sunflower head ‘Vogel’ layout design [1]. Using method-of-moments simulations of these stations, including mutual coupling, we observe a blind spot at zenith of the embedded element pattern (EEP) at 125 MHz [2], corresponding to a drop in power of around 5 dB.

This report was submitted as part of the SKA Low Frequency Aperture Array Critical Design Review describing the design of the SKA1-LOW station that took place between 2013 and 2018. The SKA1 LOW field station is inscribed in a circular area having an effective station diameter (centre to centre) of 38 meters and has 256 SKALA4 elements. This document describes the electromagnetic design of the field station. In particular it describes the layout design and the electromagnetic modelling and characteristics of the station. This document describes the effects associated with the layout and array such as mutual coupling effects, side lobe pattern and beam shape (eg. smoothness, calibration models) and presents the state of the art of our ability to measure the array performance and validate the simulation work. The current LFAA field node requirements, derived from the SKA L1 requirements, have evolved over the last years since the LFAA PDR and the System Baseline Design. The SKA1 LOW field station has been designed to meet those requirements and has therefore tracked their evolution (eg. sensitivity requirements, array diameter, etc.). The aforementioned requirements represent a very tight space with a desire for very high sensitivity over a large frequency range (7 to 1) and wide field of view (90 degrees cone around zenith) while keeping the station diameter to a minimum, so as the filling factor but at the same time allowing for sufficient space between antennas to allow for easy maintenances, amongst many others. This results in a complex design.

This work presents a workflow for simulating and processing the full-scale low-frequency telescope data of the Square Kilometre Array (SKA) Phase 1. The SKA project will enter the construction phase soon, and once completed, it will be the world's largest radio telescope and one of the world's largest data generators. The authors used Summit to mimic an endto-end SKA workflow, simulating a dataset of a typical 6 hour observation and then processing that dataset with an imaging pipeline. This workflow was deployed and run on 4,560 compute nodes, and used 27,360 GPUs to generate 2.6 PB of data. This was the first time that radio astronomical data were processed at this scale. Results show that the workflow has the capability to process one of the key SKA science cases, an Epoch of Reionization observation. This analysis also helps reveal critical design factors for the next-generation radio telescopes and the required dedicated processing facilities.

The SKA1 LOW field station is inscribed in a circular area having an effective station diameter (centre to centre) of 38 meters and has 256 SKALA4 elements. This document describes the electromagnetic design of the field station. In particular it describes the layout design and the electromagnetic modelling and characteristics of the station. This document describes the effects associated with the layout and array such as mutual coupling effects, side lobe pattern and beam shape (eg. smoothness, calibration models) and presents the state of the art of our ability to measure the array performance and validate the simulation work. The current LFAA field node requirements, derived from the SKA L1 requirements, have evolved over the last years since the LFAA PDR and the System Baseline Design. The SKA1 LOW field station has been designed to meet those requirements and has therefore tracked their evolution (eg. sensitivity requirements, array diameter, etc.). The aforementioned requirements represent a very tight space with a desire for very high sensitivity over a large frequency range (7 to 1) and wide field of view (90 degrees cone around zenith) while keeping the station diameter to a minimum, so as the filling factor but at the same time allowing for sufficient space between antennas to allow for easy maintenances, amongst many others. This results in a complex design.

The Square Kilometre Array (SKA) will be the largest radio telescope constructed to date and the largest Big Data project in the known Universe. The first phase of the project will generate 160 terabytes every second. This amounts to 5 zettabytes (5 million petabytes) of data that will be generated by the facility each year - a data rate equivalent to 5 times the estimated global internet traffic in 2015. These data need to be reduced and then continuously ingested by the SKA Science Data Processor (SDP). Within the SDP Consortium, we are contributing to various roles in the development of the telescope including building a lightweight end-to-end prototype of the major components of the SDP system - a project we call the SDP Integration Prototype (SIP). The aim is to build a mini, fully-operational SDP, for which we have been developing realistic SKA-like science pipelines that can handle these unprecedented data volumes.

The Square Kilometre Array (SKA) will be both the largest radio telescope ever constructed and the largest Big Data project in the known Universe. The first phase of the project will generate on the order of 5 zettabytes of data per year. A critical task for the SKA will be its ability to process data for science, which will need to be conducted by science pipelines. Together with polarization data from the LOFAR Multifrequency Snapshot Sky Survey (MSSS), we have been developing a realistic SKA-like science pipeline that can handle the large data volumes generated by LOFAR at 150 MHz. The pipeline uses task-based parallelism to image, detect sources, and perform Faraday Tomography across the entire LOFAR sky. The project thereby provides a unique opportunity to contribute to the technological development of the SKA telescope, while simultaneously enabling cutting-edge scientific results. In this paper, we provide an update on current efforts to develop a science pipeline that can enable tight constraints on the magnetised large-scale structure of the Universe.

The Square Kilometre Array's Low Frequency instrument (SKA-LOW) will be the most sensitive aperture array ever used for radio astronomy, and will operate in the under-sampled regime for most of the frequency band where grating-lobes pose particular challenges. To achieve the expected level of sensitivity for SKA-LOW, it is particularly important to understand how interfering sources in both near and far side-lobes of the station beam affect the imaging performance. We discuss options for station designs, and adopting a random element layout, we assess its effectiveness by investigating how sources far from the main lobe of the station beam degrade images of the target field. These sources have the effect of introducing a noise-like corruption to images, which we call the Far Side-lobe Source Noise (FSSN). Using OSKAR, a GPU-accelerated software simulator, we carried out end-to-end simulations using an all-sky model and telescope configuration representative of the SKA-LOW instrument. The FSSN is a function of both the station beam and the interferometric point spread function, and decreases with increasing observation time until the coverage of the aperture plane no longer improves. Using apodisation to reduce the level of near-in side-lobes of the station beam had a noticeable improvement on the level of FSSN at low frequencies. Our results indicate that the effects of picking up sources in the side-lobes are worse at low frequencies, where the array is less sparse.

The removal of the Galactic and extragalactic foregrounds remains a major challenge for those wishing to make a detection of the Epoch of Reionization (EoR) 21 cm signal. Multiple methods of modelling these foregrounds with varying levels of assumption have been trialled and shown promising recoveries on simulated data. Recently however there has been increased discussion of using the expected shape of the foregrounds in Fourier space to define an EoR window free of foreground contamination. By carrying out analysis within this window only, one can avoid the foregrounds and any statistical bias they might introduce by instead removing these foregrounds. In this paper, we discuss the advantages and disadvantages of both foreground removal and foreground avoidance. We create a series of simulations with noise levels in line with both current and future experiments and compare the recovered statistical cosmological signal from foreground avoidance and a simplified, frequency independent foreground removal model. We find that for current generation experiments, while foreground avoidance enables a better recovery at k(perp) > 0.6Mpc(-1), foreground removal is able to recover significantly more signal at small k(los) for both current and future experiments. We also relax the assumption that the foregrounds are smooth. For line-of-sight variations only, foreground removal is able to make a good signal recovery even at 1 per cent while foreground avoidance is compromised significantly. We find that both methods perform well for foreground models with line-of-sight and spatial variations around 0.1 per cent however at levels larger than this both methods are compromised.

The exceptional sensitivity of the SKA will allow observations of the Cosmic Dawn and Epoch of Reionization (CD/EoR) in unprecedented detail, both spectrally and spatially. This wealth of information is buried under Galactic and extragalactic foregrounds, which must be removed accurately and precisely in order to reveal the cosmological signal. This problem has been addressed already for the previous generation of radio telescopes, but the application to SKA is different in many aspects. In this chapter we summarise the contributions to the field of foreground removal in the context of high redshift and high sensitivity 21-cm measurements. We use a state-of-the-art simulation of the SKA Phase 1 observations complete with cosmological signal, foregrounds and frequency-dependent instrumental effects to test both parametric and non-parametric foreground removal methods. We compare the recovered cosmological signal using several different statistics and explore one of the most exciting possibilities with the SKA --- imaging of the ionized bubbles. We find that with current methods it is possible to remove the foregrounds with great accuracy and to get impressive power spectra and images of the cosmological signal. The frequency-dependent PSF of the instrument complicates this recovery, so we resort to splitting the observation bandwidth into smaller segments, each of a common resolution. If the foregrounds are allowed a random variation from the smooth power law along the line of sight, methods exploiting the smoothness of foregrounds or a parametrization of their behaviour are challenged much more than non-parametric ones. However, we show that correction techniques can be implemented to restore the performances of parametric approaches, as long as the first-order approximation of a power law stands.

Simulations of SKA1-low were performed to estimate the noise level in images produced by the telescope over a frequency range 50-600 MHz, which extends the 50-350 MHz range of the current baseline design. The root-mean-square (RMS) deviation between images produced by an ideal, error-free SKA1-low and those produced by SKA1-low with varying levels of uncorrelated gain and phase errors was simulated. The residual in-field and sidelobe noise levels were assessed. It was found that the RMS deviations decreased as the frequency increased. The residual sidelobe noise decreased by a factor of ~5 from 50 to 100 MHz, and continued to decrease at higher frequencies, attributable to wider strong sidelobes and brighter sources at lower frequencies. The thermal noise limit is found to range between ~10 - 0.3 $μ$Jy and is reached after ~100-100 000 hrs integration, depending on observation frequency, with the shortest integration time required at ~100 MHz.

In this paper we show how phase errors between different analogue signal channels of a phased array affects the quality of the output phased array beam. Specifically we look at the reduction in forward gain of the array beam and the array beam pointing offset as a function of phase errors and array size. We use a combination of simulations made using the aperture array simulator OSKAR, developed at the University of Oxford, in conjunction with measured results taken from 2-PAD; a functioning, astronomical, dual-polarisation, digital beamforming, 4 × 4 element, aperture array prototype (Greenwood 2007) for the Square Kilometre Array (Taylor 2007) developed by a consortium of UK universities. We show that phase errors between different signal channels is specifically an important issue for broadband arrays like 2-PAD, and comment on the relative benefit of digital beamforming versus analogue beamforming engines.