High-Performance Radio Telescope Array Data Processing Framework

As radio telescope projects grow larger with more antennas observing wider bandwidths, data rates are rapidly increasing. The Square Kilometer Array and other next generation observatories will usher in an era of exascale data and beyond [1]. This necessitates an equivalent increase in data transfers, processing speeds, and emphasis on real-time analysis. Reducing the performance gap between high-level science algorithm development (frequently in Python) and real-time, production code would allow astronomers to better utilize the hardware available to them. To enable this, we create a high-level array data processing pipeline framework in the Julia programming language, featuring templates for modular data processing algorithms. We demonstrate its performance with a spectral kurtosis algorithm and show that the new interface does not introduce significant processing overhead. In future work, we will explore the signal processing potential of new hardware accelerators present in modern GPUs. Such accelerators promise improved performance along with new programming challenges.