Studying the redshifted 21-cm signal from the neutral hydrogen during the Epoch of Reionisation and Cosmic Dawn is fundamental for understanding the physics of the early universe. One of the challenges that 21-cm experiments face is the contamination by bright foreground sources, such as Cygnus A, for which accurate spatial and spectral models are needed to minimise the residual contamination after their removal. In this work, we develop a new, high-resolution model of Cygnus A using Low Frequency Array (LOFAR) observations in the 110-250 MHz range, improving upon previous models by incorporating physical spectral information through the forced-spectrum method during multi-frequency deconvolution. This approach addresses the limitations of earlier models by providing a more accurate representation of the complex structure and spectral behaviour of Cygnus A, including the spectral turnover in its brightest hotspots. The impact of this new model on the LOFAR 21-cm signal power spectrum is assessed by comparing it with both simulated and observed North Celestial Pole datasets. Significant improvements are observed in the cylindrical power spectrum along the Cygnus A direction, highlighting the importance of having spectrally accurate models of the brightest foreground sources. However, this improvement is washed out in the spherical power spectrum, where we measure differences of a few hundred mK at k < 0.63 h cMpc(-1), but not statistically significant. The results suggest that other systematic effects must be mitigated before a substantial impact on 21-cm power spectrum can be achieved.
One of the main obstacles preventing the detection of the redshifted 21 cm signal from neutral hydrogen in the early Universe is the astrophysical foreground emission, which is several orders of magnitude brighter than the signal. The foregrounds, due to their smooth spectra, are expected to predominantly occupy a region in the cylindrical power spectrum known as the foreground wedge. However, the conventional equations describing the extent of the foreground wedge are derived under a flat-sky approximation. This assumption breaks down for tracking wide-field instruments, thus rendering these equations inapplicable in these situations. In this paper we derive equations for the full sky foreground wedge, and show that the foregrounds can potentially extend far beyond what the conventional equations suggest. We also derive the equations that describe a specific bright source in the cylindrical power spectrum space. The validity of both sets of equations is tested against numerical simulations. Many current and upcoming interferometers (e.g., LOFAR, NenuFAR, MWA, SKA) are wide-field phase-tracking instruments. These equations give us new insights into the nature of foreground contamination in the cylindrical power spectra estimated using wide-field instruments. Additionally, they allow us to accurately associate features in the power spectrum with foregrounds or instrumental effects. The equations are also important for correctly selecting the epoch of reionization (EoR) window for foreground avoidance analyses, and for planning 21 cm observations. In future analyses, we recommend using these updated horizon lines to indicate the foreground wedge in the cylindrical power spectrum accurately. The new equations for generating the updated wedge lines are made available in the Python library, pslines.
Modern self-calibration pipelines can produce high-quality radio interferometric imaging results at the cost of significant computational effort. The computational cost is expected to increase even further with the data volumes produced by the Square Kilometre Array (SKA). We are developing, improving and maintaining a code base for self-calibration and imaging. To identify performance improvements, we perform benchmarking on a quarterly cadence, In this contribution, we present the becnhmarking results from February 2024 and February 2025 and discuss the improvements made to achieve that progress as well as our plans for the near future.
The origin of the radio synchrotron background (RSB) is currently unknown. Its understanding might have profound implications in fundamental physics or might reveal a new class of radio emitters. In this work, we consider the scenario in which the RSB is due to extragalactic radio sources and measure the angular cross-correlation of LOFAR images of the diffuse radio sky with matter tracers at different redshifts, provided by galaxy catalogs and CMB lensing. We compare these measured cross-correlations to those expected for models of RSB sources. We find that low-redshift populations of discrete sources are excluded by the data, while higher redshift explanations are compatible with available observations. We also conclude that at least 20% of the RSB surface brightness level must originate from populations tracing the large-scale distribution of matter in the universe, indicating that at least this fraction of the RSB is of extragalactic origin. Future measurements of the correlation between the RSB and tracers of high-redshift sources will be crucial to constraining the source population of the RSB.
We present the deepest wide-field 115–166 MHz image at sub-arcsecond resolution spanning an area of 2.5° × 2.5° centred at the ELAIS-N1 deep field. To achieve this, we improved the direction-independent (DI) and direction-dependent (DD) calibrations for the International LOw Frequency ARray (LOFAR) Telescope. This enhancement enabled us to efficiently process 32 h of data from four different 8-h observations using the high-band antennas (HBAs) of all 52 stations, covering baselines up to approximately 2000 km across Europe. The DI calibration was improved by using an accurate sky model and refining the series of calibration steps on the in-field calibrator, while the DD calibration was improved by adopting a more automated approach for selecting the DD calibrators and inspecting the self-calibration on these sources. For our brightest calibrators, we also added an additional round of self-calibration for the Dutch core and remote stations in order to refine the solutions for shorter baselines. To complement our highest resolution at 0.3″, we also made intermediate resolution wide-field images at 0.6″ and 1.2″. Our resulting wide-field images achieve a central noise level of 14 μJy beam−1 at 0.3″, doubling the depth and uncovering four times more objects than the Lockman Hole deep field image at comparable resolution but with only 8 h of data. Compared to LOFAR imaging without the international stations, we note that due to the increased collecting area and the absence of confusion noise, we reached a point-source sensitivity comparable to a 500-h ELAIS-N1 6″ image with 16 times less observing time. Importantly, we have found that the computing costs for the same amount of data are almost halved (to about 139 000 CPU h per 8 h of data) compared to previous efforts, though they remain high. Our work underscores the value and feasibility of exploiting all Dutch and international LOFAR stations to make deep wide-field images at sub-arcsecond resolution.
The data production rate of future telescopes like LOFAR2.0 and SKA will require processing pipelines with high computational performace and a large degree of automation and configurability. In this contribution, we show how we are addressing these challenges with a combination of high-performance processing components, primarily written in C++, and a Python-based pipeline script orchestrating the work of the processing components. We present the current status and sketch future developments.
The turbulent ionosphere causes phase shifts to incoming radio waves on a broad range of temporal and spatial scales. When an interferometer is not sufficiently calibrated for the direction-dependent ionospheric effects, the time-varying phase shifts can cause the signal to decorrelate. The ionosphere's influence over various spatiotemporal scales introduces a baseline-dependent effect on the interferometric array. We study the impact of baseline-dependent decorrelation on high-redshift observations with the Low Frequency Array (LOFAR). Datasets with a range of ionospheric corruptions are simulated using a thin-screen ionosphere model, and calibrated using the state-of-the-art LOFAR Epoch of Reionisation pipeline. For the first time ever, we show the ionospheric impact on various stages of the calibration process including an analysis of the transfer of gain errors from longer to shorter baselines using realistic end-to-end simulations. We find that direction-dependent calibration for source subtraction leaves excess power of up to two orders of magnitude above the thermal noise at the largest spectral scales in the cylindrically averaged auto-power spectrum under normal ionospheric conditions. However, we demonstrate that this excess power can be removed through Gaussian process regression, leaving no excess power above the ten per cent level for a 5km diffractive scale. We conclude that ionospheric errors, in the absence of interactions with other aggravating effects, do not constitute a dominant component in the excess power observed in LOFAR Epoch of Reionisation observations of the North Celestial Pole. Future work should therefore focus on less spectrally smooth effects, such as beam modelling errors.
Measurement of the redshifted 21-cm signal of neutral hydrogen from the Cosmic Dawn (CD) and Epoch of Reionisation (EoR) promises to unveil a wealth of information about the astrophysical processes during the first billion years of evolution of the universe. The AARTFAAC Cosmic Explorer (ACE) utilises the AARTFAAC wide-field imager of LOFAR to measure the power spectrum of the intensity fluctuations of the redshifted 21-cm signal from the CD at z~18. The RFI from various sources contaminates the observed data and it is crucial to exclude the RFI-affected data in the analysis for reliable detection. In this work, we investigate the impact of non-ground-based transient RFI using cross-power spectra and cross-coherence metrics to assess the correlation of RFI over time and investigate the level of impact of transient RFI on the ACE 21-cm power spectrum estimation. We detected moving sky-based transient RFI sources that cross the field of view within a few minutes and appear to be mainly from aeroplane communication beacons at the location of the LOFAR core in the 72-75 MHz band, by inspecting filtered images. This transient RFI is mostly uncorrelated over time and is only expected to dominate over the thermal noise for an extremely deep integration time of 3000 hours or more with a hypothetical instrument that is sky temperature dominated at 75 MHz. We find no visible correlation over different k-modes in Fourier space in the presence of noise for realistic thermal noise scenarios. We conclude that the sky-based transient RFI from aeroplanes, satellites and meteorites at present does not pose a significant concern for the ACE analyses at the current level of sensitivity and after integrating over the available 500 hours of observed data. However, it is crucial to mitigate or filter such transient RFI for more sensitive experiments aiming for significantly deeper integration.
Context. Current radio interferometers output multi-petabyte-scale volumes of data per year, making the storage, transfer, and processing of these data a sizeable challenge. This challenge is expected to grow with next-generation telescopes such as the Square Kilometre Array (SKA), which will produce a considerably larger data volume than current instruments. Lossy compression of interferometric data post-correlation can abate this challenge, but any drawbacks from the compression should be well understood in advance. Aims. Lossy data compression reduces the precision of data, introducing additional noise. Since high-redshift (e.g., cosmic dawn or epoch of reionization) 21 cm studies impose strict precision requirements, the impact of this effect on the 21 cm signal power spectrum statistic is investigated in a bid to rule out unwanted systematics. Methods. We applied DYSCO visibility compression, a technique for normalizing and quantizing specifically designed for radio interferometric data, to observed visibilities datasets from the LOFAR telescope as well as simulated ones. The power spectrum of these data was analyzed, and we establish the level of the compression noise in the power spectrum in comparison to the thermal noise. We also examined its coherency behavior by employing the cross-coherence metric. Finally, for optimal compression results, we compared the compression noise obtained from different compression settings to a nominal 21 cm signal power. Results. From a single night of observation, we find that the noise introduced due to the compression is more than five orders of magnitude lower than the thermal noise level in the power spectrum. The noise does not affect calibration. Furthermore, the noise remains subdominant to the noise introduced by the nonlinear calibration algorithm used following random parameter initialization across different runs. The compression noise shows no correlation with the sky signal and has no measurable coherent component, therefore averaging down optimally with the integration of more data. The level of compression error in the power spectrum ultimately depends on the compression settings. Conclusions. DYSCO visibility compression is found to be an insignificant concern for 21 cm power spectrum studies. Hence, data volumes can be safely reduced by factors of similar to 4 with insignificant bias to the final power spectrum. Data from SKA-Low will likely be compressible by the same factor as data from LOFAR owing to the similarities of the two instruments. The same technique can be used to compress data from other telescopes, but a small adjustment of the compression parameters might be required.
The redshifted 21 cm signal from neutral hydrogen is a direct probe of the physics of the early universe and has been an important science driver of many present and upcoming radio interferometers. In this study we use a single night of observations with the New Extension in Nan cay Upgrading LOFAR (NenuFAR) to place upper limits on the 21 cm power spectrum from cosmic dawn at a redshift of $z$ = 20.3. NenuFAR is a new low-frequency radio interferometer, operating in the 10--85 MHz frequency range, currently under construction at the Nan cay Radio Observatory in France. It is a phased array instrument with a very dense $u coverage at short baselines, making it one of the most sensitive instruments for 21 cm cosmology analyses at these frequencies. Our analysis adopts the foreground subtraction approach, in which sky sources are modeled and subtracted through calibration and residual foregrounds are subsequently removed using Gaussian process regression. The final power spectra are constructed from the gridded residual data cubes in the $u plane. Signal injection tests are performed at each step of the analysis pipeline, the relevant pipeline settings are optimized to ensure minimal signal loss, and any signal suppression is accounted for through a bias correction on our final upper limits. We obtain a best 2sigma upper limit of $2.4 mK $ at $z$ = 20.3 and $k$ = 0.041 $h\ cMpc $. We see a strong excess power in the data, making our upper limits two orders of magnitude higher than the thermal noise limit. We investigate the origin and nature of this excess power and discuss further improvements to the analysis pipeline that can potentially mitigate it and consequently allow us to reach thermal noise sensitivity when multiple nights of observations are processed in the future.
Context.Measurement of the highly redshifted and faint 21-cm signal of neutral hydrogen from the Cosmic Dawn and Epoch of Reionisation promises to unveil a wealth of information about the astrophysical processes that governed the structure formation and evolution of the universe during the first billion years of its evolution.Aims.The AARTFAAC Cosmic Explorer (ACE) program utilises the AARTFAAC wide-field imager of LOFAR to measure the power spectrum of the intensity fluctuations of the redshifted 21-cm signal from the Cosmic Dawn atz ∼ 18 corresponding to the global 21-cm absorption feature reported by the EDGES experiment. Radio frequency interference (RFI) from various sources, such as aeroplane communication, contaminates the observed data and it is crucial to exclude the RFI-affected data in the analysis for any reliable detection. In this work, we solely focus on investigating the impact of non-ground-based transient RFI on the analysis of ACE observations.Methods.We use cross-power spectra and cross-coherence metrics to assess the correlation of RFI over time and investigate the level of impact of transient RFI on the 21-cm signal power spectrum estimation.Results.We detected moving sky-based transient RFI sources that cross the field of view within a few minutes and appear to be mainly from aeroplane communication beacons at the location of the LOFAR core in the 72−75 MHz band (a part of the EDGES absorption trough), by inspecting filtered images. We find that this transient RFI is mostly uncorrelated over time and is only expected to dominate over the thermal noise for an extremely deep integration time of 3000 h or more with a hypothetical instrument that is sky temperature dominated at 75 MHz. We find no visible correlation over differentk-modes in Fourier space in the presence of noise for realistic thermal noise scenarios.Conclusions.We conclude that the sky-based transient RFI from aeroplanes, satellites and meteorites at present does not pose a significant concern for the ACE analyses at the current level of sensitivity and after integrating over the available ∼500 h of observed data. However, it is crucial to mitigate or filter such transient RFI for more sensitive experiments aiming for significantly deeper integration.
We present the largest low frequency (120~MHz) arcminute resolution image of the radio synchrotron background (RSB) to date, and its corresponding angular power spectrum of anisotropies (APS) with angular scales ranging from $3^\circ$ to $0.3^\prime$. We show that the RSB around the North Celestial Pole has a significant excess anisotropy power at all scales over a model of unclustered point sources based on source counts of known source classes. This anisotropy excess, which does not seem attributable to the diffuse Galactic emission, could be linked to the surface brightness excess of the RSB. To better understand the information contained within the measured APS, we model the RSB varying the brightness distribution, size, and angular clustering of potential sources. We show that the observed APS could be produced by a population of faint clustered point sources only if the clustering is extreme and the size of the Gaussian clusters is $\lesssim 1'$. We also show that the observed APS could be produced by a population of faint diffuse sources with sizes $\lesssim 1'$, and this is supported by features present in our image. Both of these cases would also cause an associated surface brightness excess. These classes of sources are in a parameter space not well probed by even the deepest radio surveys to date.
Context. Apertif is a multi-beam receiver system for the Westerbork Synthesis Radio Telescope that operates at 1.1-1.5 GHz, which overlaps with various radio services, resulting in contamination of astronomical signals with radio-frequency interference (RFI). Aims. We analyze approaches to mitigate Apertif interference and design an automated detection procedure for its imaging mode. Using this approach, we present long-term RFI detection results of over 300 Apertif observations. Methods. Our approach is based on the AOFlagger detection approach. We introduce several new features, including ways to deal with ranges of invalid data (e.g. caused by shadowing) in both the SumThreshold and scale-invariant rank operator steps; pre-calibration bandpass calibration; auto-correlation flagging; and HI flagging avoidance. These methods are implemented in a new framework that uses the Lua language for scripting, which is new in AOFlagger version 3. Results. Our approach removes RFI fully automatically, and is robust and effective enough for further calibration and (continuum) imaging of these data. Analysis of 304 observations show an average of 11.1% of lost data due to RFI with a large spread. We observe 14.6% RFI in auto-correlations. Computationally, AOFlagger achieves a throughput of 370 MB/s on a single computing node. Compared to published machine learning results, the method is one to two orders of magnitude faster.
We present a new method, called "forced-spectrum fitting", for physically-based spectral modelling of radio sources during deconvolution. This improves upon current common deconvolution fitting methods, which often produce inaccurate spectra. Our method uses any pre-existing spectral index map to assign spectral indices to each model component cleaned during the multi-frequency deconvolution of WSClean, where the pre-determined spectrum is fitted. The component magnitude is evaluated by performing a modified weighted linear least-squares fit. We test this method on a simulated LOFAR-HBA observation of the 3C196 QSO and a real LOFAR-HBA observation of the 4C+55.16 FRI galaxy. We compare the results from the forced-spectrum fitting with traditional joined-channel deconvolution using polynomial fitting. Because no prior spectral information was available for 4C+55.16, we demonstrate a method for extracting spectral indices in the observed frequency band using "clustering". The models generated by the forced-spectrum fitting are used to improve the calibration of the datasets. The final residuals are comparable to existing multi-frequency deconvolution methods, but the output model agrees with the provided spectral index map, embedding correct spectral information. While forced-spectrum fitting does not solve the determination of the spectral information itself, it enables the construction of accurate multi-frequency models that can be used for wide-band calibration and subtraction.
We present a new image of a 26.5 square degree region in the Bo\"otes constellation obtained at 1.4 GHz using the Aperture Tile in Focus (Apertif) system on the Westerbork Synthesis Radio Telescope. We use a newly developed processing pipeline which includes direction-dependent self-calibration which provides a significant improvement of the quality of the images compared to those released as part of the Apertif first data release. For the Bo\"otes region, we mosaic 187 Apertif images and extract a source catalog. The mosaic image has an angular resolution of 27${\times}$11.5 arcseconds and a median background noise of 40 ${\mu}$Jy/beam. The catalog has 8994 sources and is complete down to the 0.3 mJy level. We combine the Apertif image with LOFAR images of the Bo\"otes field at 54 and 150 MHz to study spectral properties of the sources. We find a spectral flattening towards low flux density sources. Using the spectral index limits from Apertif non-detections we derive that up to 9 percent of the sources have ultra-steep spectra with a slope steeper than -1.2. Steepening of the spectral index with increasing redshift is also seen in the data showing a different dependency for the low-frequency spectral index and the high frequency one. This can be explained by a population of sources having concave radio spectra with a turnover frequency around the LOFAR band. Additionally, we discuss cases of individual extended sources with an interesting resolved spectral structure. With the improved pipeline, we aim to continue processing data from the Apertif wide-area surveys and release the improved 1.4 GHz images of several famous fields.
The Low Frequency Array (LOFAR) is the only existing radio interferometer able to observe at ultra-low frequencies (<100 MHz) with high resolution (<15") and high sensitivity (<1 mJy/beam). To exploit these capabilities, the LOFAR Surveys Key Science Project is using the LOFAR Low Band Antenna (LBA) to carry out a sensitive wide-area survey at 41-66 MHz named the LOFAR LBA Sky Survey (LoLSS). LoLSS is covering the whole northern sky above declination 24 deg with a resolution of 15" and a sensitivity of 1-2 mJy/beam (1 sigma) depending on declination, field properties, and observing conditions. Here we present the first data release. An automated pipeline was used to reduce the 95 fields included in this data release. The data reduction procedures developed for this project have general application and are currently being used to process LOFAR LBA interferometric observations. Compared to the preliminary release, direction-dependent errors have been corrected for during the calibration process. This results in a typical sensitivity of 1.55 mJy/beam at the target resolution of 15". The first data release of the LOFAR LBA Sky Survey covers 650 sqdeg in the HETDEX spring field. The resultant data products released to the community include mosaic images (I and V Stokes) of the region, and a catalogue of 42463 detected sources and related Gaussian components used to describe sources' morphologies. Separate catalogues for 6 in-band frequencies are also released. The first data release of LoLSS shows that, despite the influences of the ionosphere, LOFAR can conduct large-scale surveys in the frequency window 42-66 MHz with unprecedentedly high sensitivity and resolution. The data can be used to derive unique information on the low-frequency spectral properties of many thousands of sources with a wide range of applications in extragalactic and galactic astronomy.
Detecting the 21-cm signal from the Epoch of Reionisation (EoR) is challenging due to the strong astrophysical foregrounds, ionospheric effects, radio frequency interference and instrumental effects. Understanding and calibrating these effects are crucial for the detection. In this work, we introduce a newly developed direction-dependent (DD) calibration algorithm DDECAL and compare its performance with an existing algorithm, SAGECAL, in the context of the LOFAR-EoR 21-cm power spectrum experiment. In our data set, the North Celestial Pole (NCP) and its flanking fields were observed simultaneously. We analyse the NCP and one of its flanking fields. The NCP field is calibrated by the standard pipeline, using SAGECAL with an extensive sky model and 122 directions, and the flanking field is calibrated by DDECAL and SAGECAL with a simpler sky model and 22 directions. Additionally, two strategies are used for subtracting Cassiopeia A and Cygnus A. The results show that DDECAL performs better at subtracting sources in the primary beam region due to the application of a beam model, while SAGECAL performs better at subtracting Cassiopeia A and Cygnus A. This indicates that including a beam model during DD calibration significantly improves the performance. The benefit is obvious in the primary beam region. We also compare the 21-cm power spectra on two different fields. The results show that the flanking field produces better upper limits compared to the NCP in this particular observation. Despite the minor differences between DDECAL and SAGECAL due to the beam application, we find that the two algorithms yield comparable 21-cm power spectra on the LOFAR-EoR data after foreground removal. Hence, the current LOFAR-EoR 21-cm power spectrum limits are not likely to depend on the DD calibration method.
Radio spectroscopy provides a unique inspection perspective for solar and space weather research, which can reveal the plasma and energetic electron information in the solar corona and inner heliosphere. However, Radio-Frequency Interference (RFI) from human activities affects sensitive radio telescopes, and significantly affects the quality of observation. Thus, RFI detection and mitigation for the observations is necessary to obtain high quality, science-ready data. The flagging of RFI is particularly challenging for the solar and space weather observations at low frequency, because the solar radio bursts can be brighter than the RFI, and may show similar temporal behavior. In this work, we investigate RFI flagging methods for solar and space weather observations, including a strategy for AOFlagger, and a novel method that makes use of a morphology convolution. These algorithms can effectively flag RFI while preserving solar radio bursts.
For large aperture arrays like LOFAR and SKA-LOW, the point spread function (PSF) varies significantly across the wide field-of-view (FoV) of these instruments. This turns deconvolution into a computationally demanding and often labour intensive task when using standard algorithms assuming that the PSF is constant over the FoV. In this contribution, we describe an implementation of deconvolution using a direction-dependent (DD) PSF. We demonstrate the advantage of using a DD PSF on an extreme case of a LOFAR observation done with stations of two different sizes. The use of a DD PSF made convergence of deconvolution more robust thereby requiring fewer major cycles and less manual tuning to obtain a good deconvolved image.
Context. The occultation of a radio source by the plasma tail of a comet can be used to probe structure and dynamics in the tail. Such occultations are rare, and the occurrence of scintillation, due to small-scale density variations in the tail, remains somewhat controversial. Aims. A detailed observation taken with the Low-Frequency Array (LOFAR) of a serendipitous occultation of the compact radio source 3C196 by the plasma tail of comet C/2020 F3 (Neowise) is presented. 3C196 tracked almost perpendicularly behind the tail, providing a unique profile cut only a short distance downstream from the cometary nucleus itself. Methods. Interplanetary scintillation (IPS) is observed as the rapid variation of the intensity received of a compact radio source due to density variations in the solar wind. IPS in the signal received from 3C196 was observed for five hours, covering the full transit behind the plasma tail of comet C/2020 F3 (Neowise) on 16 July 2020, and allowing an assessment of the solar wind in which the comet and its tail are embedded. Results. The results reveal a sudden and strong enhancement in scintillation which is unequivocally attributable to the plasma tail. The strongest scintillation is associated with the tail boundaries, weaker scintillation is seen within the tail, and previously-unreported periodic variations in scintillation are noted, possibly associated with individual filaments of plasma. Furthermore, contributions from the solar wind and comet tail are separated to measure a sharp decrease in the velocity of material within the tail, suggesting a steep velocity shear resulting in strong turbulence along the tail boundary.