FACT-Tools - Processing High-Volume Telescope Data

The amount of data produced in astroparticle physics exceeds the capacities of traditional data processing systems. Within the Big Data era, the field of computer science has brought up a plethora of tools and methods to scale data processing to a large number of nodes. In the stress field of rapid prototyping scientific algorithms and the implementation of scalable processing pipelines, we propose an abstraction for high-level modelling of such pipelines. This enables physicists to concentrate on their domain specific use-case at hand, while maintaining the benefit to deploy their algorithms within a scalable execution platform. In this paper, we outline the collaborative work of physicists and computer scientists to develop a modeling framework that fits the aforementioned setting. The framework is used within the FACT project, showing its real-world practicability and ease-of-use.