Stuart J. Mumford∗1, 2, 3, Nabil Freij4, Steven Christe5, Jack Ireland5, Florian Mayer6, V. Keith Hughitt7, Albert Y. Shih5, Daniel F. Ryan8, 5, Simon Liedtke6, David Pérez-Suárez9, Pritish Chakraborty10, Vishnunarayan K I.6, Andrew Inglis11, Punyaslok Pattnaik12, Brigitta Sipőcz13, Rishabh Sharma6, Andrew Leonard3, David Stansby14, Russell Hewett15, Alex Hamilton6, Laura Hayes5, Asish Panda6, Matt Earnshaw6, Nitin Choudhary16, Ankit Kumar6, Prateek Chanda17, Md Akramul Haque18, Michael S Kirk11, Michael Mueller6, Sudarshan Konge6, Rajul Srivastava6, Yash Jain19, Samuel Bennett6, Ankit Baruah6, Will Barnes20, Michael Charlton6, Shane Maloney21, Nicky Chorley22, Himanshu6, Sanskar Modi6, James Paul Mason6, Naman96396, Jose Ivan Campos Rozo23, Larry Manley6, Agneet Chatterjee24, John Evans6, Michael Malocha6, Monica G. Bobra25, Sourav Ghosh24, Airmansmith976, Dominik Stańczak26, Ruben De Visscher6, Shresth Verma27, Ankit Agrawal6, Dumindu Buddhika6, Swapnil Sharma6, Jongyeob Park28, Matt Bates6, Dhruv Goel6, Garrison Taylor29, Goran Cetusic6, Jacob6, Mateo Inchaurrandieta6, Sally Dacie30, Sanjeev Dubey6, Deepankar Sharma6, Erik M. Bray6, Jai Ram Rideout31, Serge Zahniy5, Tomas Meszaros6, Abhigyan Bose6, André Chicrala32, Ankit6, Chloé Guennou6, Daniel D’Avella6, Daniel Williams33, Jordan Ballew6, Nick Murphy34, Priyank Lodha6, Thomas Robitaille6, Yash Krishan6, Andrew Hill6, Arthur Eigenbrot35, Benjamin Mampaey36, Bernhard M. Wiedemann6, Carlos Molina6, Duygu Keşkek6, Ishtyaq Habib6, Joseph Letts6, Juanjo Bazán37, Quinn Arbolante38, Reid Gomillion6, Yash Kothari6, Yash Sharma6, Abigail L. Stevens39, 40, Adrian Price-Whelan41, Ambar Mehrotra6, Arseniy Kustov6, Brandon Stone6, Trung Kien Dang42, Emmanuel Arias6, Fionnlagh Mackenzie Dover1, Freek Verstringe36, Gulshan Kumar43, Harsh Mathur44, Igor Babuschkin6, Jaylen Wimbish6, Juan Camilo Buitrago-Casas6, Kalpesh Krishna45, Kaustubh Hiware46, Manas Mangaonkar6, Matthew Mendero6, Mickaël Schoentgen6, Norbert G Gyenge47, Ole Streicher48, Rajasekhar Reddy Mekala6, Rishabh Mishra6, Shashank Srikanth43, Sarthak Jain6, Tannmay Yadav49, Tessa D. Wilkinson6, Tiago M. D. Pereira50, 51, Yudhik Agrawal12, jamescalixto6, yasintoda6, and Sophie A. Murray52
The Astropy project supports and fosters the development of open-source and openly-developed Python packages that provide commonly-needed functionality to the astronomical community. A key element of the Astropy project is the core package Astropy, which serves as the foundation for more specialized projects and packages. In this article, we provide an overview of the organization of the Astropy project and summarize key features in the core package as of the recent major release, version 2.0. We then describe the project infrastructure designed to facilitate and support development for a broader ecosystem of inter-operable packages. We conclude with a future outlook of planned new features and directions for the broader Astropy project.
The Flexible Image Transport System (FITS) standard has been a great boon to astronomy, allowing observatories, scientists and the public to exchange astronomical information easily. The FITS standard, however, is showing its age. Developed in the late 1970s, the FITS authors made a number of implementation choices that, while common at the time, are now seen to limit its utility with modern data. The authors of the FITS standard could not anticipate the challenges which we are facing today in astronomical computing. Difficulties we now face include, but are not limited to, addressing the need to handle an expanded range of specialized data product types (data models), being more conducive to the networked exchange and storage of data, handling very large datasets, and capturing significantly more complex metadata and data relationships. There are members of the community today who find some or all of these limitations unworkable, and have decided to move ahead with storing data in other formats. If this fragmentation continues, we risk abandoning the advantages of broad interoperability, and ready archivability, that the FITS format provides for astronomy. In this paper we detail some selected important problems which exist within the FITS standard today. These problems may provide insight into deeper underlying issues which reside in the format and we provide a discussion of some lessons learned. It is not our intention here to prescribe specific remedies to these issues; rather, it is to call attention of the FITS and greater astronomical computing communities to these problems in the hope that it will spur action to address them.
We present the case for developing a successor format for the immensely successful FITS format. We first review existing alternative formats and discuss why we do not believe they provide an adequate solution. The proposed format is called the Advanced Scientific Data Format (ASDF) and is based on an existing text format, YAML, that we believe removes most of the current problems with the FITS format. An overview of the capabilities of the new format is given along with specific examples. This format has the advantage that it does not limit the size of attribute names (akin to FITS keyword names) nor place restrictions on the size or type of values attributes have. Hierarchical relationships are explicit in the syntax and require no special conventions. Finally, it is capable of storing binary data within the file in its binary form. At its basic level, the format proposed has much greater applicability than for just astronomical data.
The Flexible Image Transport System (FITS) standard has been a great boon to astronomy, allowing observatories, scientists, and the public to exchange astronomical information easily. The FITS standard is, however, showing its age. Developed in the late 1970s the FITS authors made a number of implementation choices for the format that, while common at the time, are now seen to limit its utility with modern data. The authors of the FITS standard could not appreciate the challenges which we would be facing today in astronomical computing. Difficulties we now face include, but are not limited to, having to address the need to handle an expanded range of specialized data product types (data models), being more conducive to the networked exchange and storage of data, handling very large datasets and the need to capture significantly more complex metadata and data relationships.There are members of the community today who find some (or all) of these limitations unworkable, and have decided to move ahead with storing data in other formats. This reaction should be taken as a wakeup call to the FITS community to make changes in the FITS standard, or to see its usage fall. In this paper we detail some selected important problems which exist within the FITS standard today. It is not our intention to prescribe specific remedies to these issues; rather, we hope to call attention of the FITS and greater astronomical computing communities to these issues in the hopes that it will spur action to address them.
The Flexible Image Transport System (FITS) standard has been a great boon to astronomy, allowing observatories, scientists and the public to exchange astronomical information easily. The FITS standard, however, is showing its age. Developed in the late 1970s, the FITS authors made a number of implementation choices that, while common at the time, are now seen to limit its utility with modern data. The authors of the FITS standard could not anticipate the challenges which we are facing today in astronomical computing. Di culties we now face include, but are not limited to, addressing the need to handle an expanded range of specialized data product types (data models), being more conducive to the networked exchange and storage of data, handling very large datasets, and capturing significantly more complex metadata and data relationships. There are members of the community today who find some or all of these limitations unworkable, and have decided to move ahead with storing data in other formats. If this fragmentation continues, we risk abandoning the advantages of broad interoperability, and ready archivability, that the FITS format provides for astronomy. In this paper we detail some selected important problems which exist within the FITS standard today. These problems may provide insight into deeper underlying issues which reside in the format and we provide a discussion of some lessons learned. It is not our intention here to prescribe specific remedies to these issues; rather, it is to call attention of the FITS and greater astronomical computing communities to these problems in the hope that it will spur action to address them.