Aiming to assess the progress and current challenges on the formidable problem of the prediction of solar energetic events since the COSPAR / International Living With a Star (ILWS) Roadmap paper of Schrijver et al. (2015), we attempt an overview of the current status of global research efforts. By solar energetic events we refer to flares, coronal mass ejections (CMEs), and solar energetic particle (SEP) events. The emphasis, therefore, is on the prediction methods of solar flares and eruptions, as well as their associated SEP manifestations. This work complements the COSPAR International Space Weather Action Teams (ISWAT) review paper on the understanding of solar eruptions by Linton et al. (2023) (hereafter, ISWAT review papers are conventionally referred to as ’Cluster’ papers, given the ISWAT structure). Understanding solar flares and eruptions as instabilities occurring above the nominal background of solar activity is a core solar physics problem. We show that effectively predicting them stands on two pillars: physics and statistics. With statistical methods appearing at an increasing pace over the last 40 years, the last two decades have brought the critical realization that data science needs to be involved, as well, as volumes of diverse ground- and space-based data give rise to a Big Data landscape that cannot be handled, let alone processed, with conventional statistics. Dimensionality reduction in immense parameter spaces with the dual aim of both interpreting and forecasting solar energetic events has brought artificial intelligence (AI) methodologies, in variants of machine and deep learning, developed particularly for tackling Big Data problems. With interdisciplinarity firmly present, we outline an envisioned framework on which statistical and AI methodologies should be verified in terms of performance and validated against each other. We emphasize that a homogenized and streamlined (i.e., readily performed) method validation is another open challenge. The performance of the plethora of methods is typically far from perfect, with physical reasons to blame, besides practical shortcomings: imperfect data, data gaps and a lack of multiple, and meaningful, vantage points of solar observations. We briefly discuss these issues, too, that shape our desired short- and long-term objectives for an efficient future predictive capability. A central aim of this article is to trigger meaningful, targeted discussions that will compel the community to adopt standards for performance verification and validation, which could be maintained and enriched by institutions such as NASA’s Community Coordinated Modeling Center (CCMC) and the community-driven COSPAR/ISWAT initiative.

We recommend that NASA maintain and fund science platforms that enable interactive and scalable data analysis in order to maximize the scientific return of data collected from space-based instruments.

We consider the flare prediction problem that distinguishes flare-imminent active regions that produce an M- or X-class flare in the succeeding 24 hr, from quiet active regions that do not produce any flares within ±24 hr. Using line-of-sight magnetograms and parameters of active regions in two data products covering Solar Cycles 23 and 24, we train and evaluate two deep learning algorithms—a convolutional neural network (CNN) and a long short-term memory (LSTM)—and their stacking ensembles. The decisions of CNN are explained using visual attribution methods. We have the following three main findings. (1) LSTM trained on data from two solar cycles achieves significantly higher true skill scores (TSSs) than that trained on data from a single solar cycle with a confidence level of at least 0.95. (2) On data from Solar Cycle 23, a stacking ensemble that combines predictions from LSTM and CNN using the TSS criterion achieves a significantly higher TSS than the “select-best” strategy with a confidence level of at least 0.95. (3) A visual attribution method called “integrated gradients” is able to attribute the CNN’s predictions of flares to the emerging magnetic flux in the active region. It also reveals a limitation of CNNs as flare prediction methods using line-of-sight magnetograms: it treats the polarity artifact of line-of-sight magnetograms as positive evidence of flares.

Three main points: 1. Data Science (DS) will be increasingly important to heliophysics; 2. Methods of heliophysics science discovery will continually evolve, requiring the use of learning technologies [e.g., machine learning (ML)] that are applied rigorously and that are capable of supporting discovery; and 3. To grow with the pace of data, technology, and workforce changes, heliophysics requires a new approach to the representation of knowledge.

In this paper, we consider incorporating data associated with the sun’s north and south polar field strengths to improve solar flare prediction performance using machine learning models. When used to supplement local data from active regions on the photospheric magnetic field of the sun, the polar field data provides global information to the predictor. While such global features have been previously proposed for predicting the next solar cycle’s intensity, in this paper we propose using them to help classify individual solar flares. We conduct experiments using HMI data employing four different machine learning algorithms that can exploit polar field information. Additionally, we propose a novel probabilistic mixture of experts model that can simply and effectively incorporate polar field data and provide on-par prediction performance with state-of-the-art solar flare prediction algorithms such as the Recurrent Neural Network (RNN). Our experimental results indicate the usefulness of the polar field data for solar flare prediction, which can improve Heidke Skill Score (HSS2) by as much as 10.1% 1 .

We use machine learning methods to predict whether an active region (AR) which produces flares will lead to a solar energetic particle (SEP) event using Space‐Weather Michelson Doppler Imager (MDI) Active Region Patches (SMARPs). This new data product is derived from maps of the solar surface magnetic field taken by the MDI aboard the Solar and Heliospheric Observatory. We survey the SMARP active regions associated with flares that appear on the solar disk between 5 June 1996 and 14 August 2010, label those that produced SEPs as positive and the rest as negative. The AR SMARP features that correspond to each flare are used to train two different types of machine learning methods, the support vector machines (SVMs) and the regression models. The results show that the SMARP data can predict whether a flare will lead to an SEP with accuracy (ACC) ≤0.72 ± 0.12 while allowing for a competitive leading time of 55.3 ± 28.6 min for forecasting the SEP events.

We present a Python tool to generate a standard dataset from solar images that allows for user-defined selection criteria and a range of pre-processing steps. Our Python tool works with all image products from both the Solar and Heliospheric Observatory (SoHO) and Solar Dynamics Observatory (SDO) missions. We discuss a dataset produced from the SoHO mission's multi-spectral images which is free of missing or corrupt data as well as planetary transits in coronagraph images, and is temporally synced making it ready for input to a machine learning system. Machine-learning-ready images are a valuable resource for the community because they can be used, for example, for forecasting space weather parameters. We illustrate the use of this data with a 3-5 day-ahead forecast of the north-south component of the interplanetary magnetic field (IMF) observed at Lagrange point one (L1). For this use case, we apply a deep convolutional neural network (CNN) to a subset of the full SoHO dataset and compare with baseline results from a Gaussian Naive Bayes classifier.

We present a new data product, called Space-Weather MDI Active Region Patches (SMARPs), derived from maps of the solar surface magnetic field taken by the Michelson Doppler Imager on board the Solar and Heliospheric Observatory. Together with the Space-Weather HMI Active Region Patches (SHARPs), derived from similar maps taken by the Helioseismic and Magnetic Imager on board the Solar Dynamics Observatory, these data provide a continuous and seamless set of maps and keywords that describe every active region observed over the last two solar cycles, from 1996 to the present day. In this paper, we describe the SMARP data and compare it to the SHARP data.

The SunPy Project developed a 13-question survey to understand the software and hardware usage of the solar-physics community. Of the solar-physics community, 364 members across 35 countries responded to our survey. We found that $99\pm 0.5$% of respondents use software in their research and 66% use the Python scientific-software stack. Students are twice as likely as faculty, staff scientists, and researchers to use Python rather than Interactive Data Language (IDL). In this respect, the astrophysics and solar-physics communities differ widely: 78% of solar-physics faculty, staff scientists, and researchers in our sample uses IDL, compared with 44% of astrophysics faculty and scientists sampled by Momcheva and Tollerud (2015). $63\pm 4$% of respondents have not taken any computer-science courses at an undergraduate or graduate level. We also found that most respondents use consumer hardware to run software for solar-physics research. Although 82% of respondents work with data from space-based or ground-based missions, some of which (e.g. the Solar Dynamics Observatory and Daniel K. Inouye Solar Telescope) produce terabytes of data a day, 14% use a regional or national cluster, 5% use a commercial cloud provider, and 29% use exclusively a laptop or desktop. Finally, we found that $73\pm 4$% of respondents cite scientific software in their research, although only $42\pm 3$% do so routinely.

One of the main science motivations for the ESA PLAnetary Transit and Oscillations (PLATO) mission is to measure exoplanet transit radii with 3% precision. In addition to flares and starspots, stellar oscillations and granulation will enforce fundamental noise floors for transiting exoplanet radius measurements. We simulate light curves of Earth-sized exoplanets transiting continuum intensity images of the Sun taken by the HMI instrument aboard SDO to investigate the uncertainties introduced on the exoplanet radius measurements by stellar granulation and oscillations. After modeling the solar variability with a Gaussian process, we find that the amplitude of solar oscillations and granulation is of order 100 ppm -- similar to the depth of an Earth transit -- and introduces a fractional uncertainty on the depth of transit of 0.73% assuming four transits are observed over the mission duration. However, when we translate the depth measurement into a radius measurement of the planet, we find a much larger radius uncertainty of 3.6%. This is due to a degeneracy between the transit radius ratio, the limb-darkening, and the impact parameter caused by the inability to constrain the transit impact parameter in the presence of stellar variability. We find that surface brightness inhomogeneity due to photospheric granulation contributes a lower limit of only 2 ppm to the photometry in-transit. The radius uncertainty due to granulation and oscillations, combined with the degeneracy with the transit impact parameter, accounts for a significant fraction of the error budget of the PLATO mission, before detector or observational noise is introduced to the light curve. If it is possible to constrain the impact parameter or to obtain follow-up observations at longer wavelengths where limb-darkening is less significant, this may enable higher precision radius measurements.

Voyager 1 and 2 observed very different boundary layers adjacent to their respective heliopause crossings. Voyager 1 observed a very thick boundary layer in the inner heliosheath while Voyager 2 observed a very thin boundary layer. Voyager 2 observed a thick magnetic barrier with enhanced total magnetic field in the inner heliosheath while Voyager 1 did not observe a similar barrier. Predicted and observed plasma properties in the inner and outer heliosheath and the magnetic shear at the heliopause crossings are used to investigate the possibility of local reconnection at the heliopause crossings. For the Voyager 1 crossing, local reconnection is suppressed. However, for the Voyager 2 crossing, the magnetic barrier reduced plasma beta in the inner heliosheath and may have facilitated local magnetic reconnection at the heliopause.

Richard Galvez , David F. Fouhey , Meng Jin , Alexandre Szenicer , Andrés Muñoz-Jaramillo , Mark C. M. Cheung , Paul J. Wright , Monica G. Bobra , Yang Liu , James Mason , and Rajat Thomas 1 Center for Data Science, New York University, New York, NY 10011, USA; richardagalvez@gmail.com 2 University of Michigan Ann Arbor, MI 48109, USA 3 Lockheed Martin Solar & Astrophysics Laboratory, Palo Alto, CA, USA 4 SETI Institute, Mountain View, CA 94043, USA 5 University of Oxford, Oxford OX1 2JD, UK 6 Southwest Research Institute, San Antonio, TX 78238, USA 7 Hansen Experimental Physics Laboratory, Stanford University, Stanford, CA 94305, USA 8 SUPA School of Physics & Astronomy, University of Glasgow, Glasgow G12 8QQ, UK 9 NASA Goddard Space Flight Center, Greenbelt, MD 20771, USA 10 University of Amsterdam, 1012 WX Amsterdam, The Netherlands Received 2020 July 17; published 2020 October 15

Stuart J. Mumford∗1, 2, 3, Nabil Freij4, Steven Christe5, Jack Ireland5, Florian Mayer6, V. Keith Hughitt7, Albert Y. Shih5, Daniel F. Ryan8, 5, Simon Liedtke6, David Pérez-Suárez9, Pritish Chakraborty10, Vishnunarayan K I.6, Andrew Inglis11, Punyaslok Pattnaik12, Brigitta Sipőcz13, Rishabh Sharma6, Andrew Leonard3, David Stansby14, Russell Hewett15, Alex Hamilton6, Laura Hayes5, Asish Panda6, Matt Earnshaw6, Nitin Choudhary16, Ankit Kumar6, Prateek Chanda17, Md Akramul Haque18, Michael S Kirk11, Michael Mueller6, Sudarshan Konge6, Rajul Srivastava6, Yash Jain19, Samuel Bennett6, Ankit Baruah6, Will Barnes20, Michael Charlton6, Shane Maloney21, Nicky Chorley22, Himanshu6, Sanskar Modi6, James Paul Mason6, Naman96396, Jose Ivan Campos Rozo23, Larry Manley6, Agneet Chatterjee24, John Evans6, Michael Malocha6, Monica G. Bobra25, Sourav Ghosh24, Airmansmith976, Dominik Stańczak26, Ruben De Visscher6, Shresth Verma27, Ankit Agrawal6, Dumindu Buddhika6, Swapnil Sharma6, Jongyeob Park28, Matt Bates6, Dhruv Goel6, Garrison Taylor29, Goran Cetusic6, Jacob6, Mateo Inchaurrandieta6, Sally Dacie30, Sanjeev Dubey6, Deepankar Sharma6, Erik M. Bray6, Jai Ram Rideout31, Serge Zahniy5, Tomas Meszaros6, Abhigyan Bose6, André Chicrala32, Ankit6, Chloé Guennou6, Daniel D’Avella6, Daniel Williams33, Jordan Ballew6, Nick Murphy34, Priyank Lodha6, Thomas Robitaille6, Yash Krishan6, Andrew Hill6, Arthur Eigenbrot35, Benjamin Mampaey36, Bernhard M. Wiedemann6, Carlos Molina6, Duygu Keşkek6, Ishtyaq Habib6, Joseph Letts6, Juanjo Bazán37, Quinn Arbolante38, Reid Gomillion6, Yash Kothari6, Yash Sharma6, Abigail L. Stevens39, 40, Adrian Price-Whelan41, Ambar Mehrotra6, Arseniy Kustov6, Brandon Stone6, Trung Kien Dang42, Emmanuel Arias6, Fionnlagh Mackenzie Dover1, Freek Verstringe36, Gulshan Kumar43, Harsh Mathur44, Igor Babuschkin6, Jaylen Wimbish6, Juan Camilo Buitrago-Casas6, Kalpesh Krishna45, Kaustubh Hiware46, Manas Mangaonkar6, Matthew Mendero6, Mickaël Schoentgen6, Norbert G Gyenge47, Ole Streicher48, Rajasekhar Reddy Mekala6, Rishabh Mishra6, Shashank Srikanth43, Sarthak Jain6, Tannmay Yadav49, Tessa D. Wilkinson6, Tiago M. D. Pereira50, 51, Yudhik Agrawal12, jamescalixto6, yasintoda6, and Sophie A. Murray52

The goal of the SunPy project is to facilitate and promote the use and development of community-led, free, and open source data analysis software for solar physics based on the scientific Python environment. The project achieves this goal by developing and maintaining the sunpy core package and supporting an ecosystem of affiliated packages. This paper describes the first official stable release (version 1.0) of the core package, as well as the project organization and infrastructure. This paper concludes with a discussion of the future of the SunPy project.

1 National Research Council Postdoctoral Research Associate residing at the Naval Research Laboratory, Washington, D.C. 20375, USA 2 Lockheed Martin Solar and Astrophysics Laboratory, Palo Alto, CA 94304, USA 3 W. W. Hansen Experimental Physics Laboratory, Stanford University, Stanford, CA 94305, USA 4 University Corporation for Atmospheric Research, Boulder, CO 80301, USA 5 Aperio Software Ltd, Leeds LS6 3HN, UK 6 School of Mathematics and Statistics, The University of Sheffield, Sheffield S3 7RH, UK 7 Princeton University, Princeton, NJ 08544, USA 8 NASA Goddard Space Flight Center, Greenbelt, MD 20771, USA 9 Mullard Space Science Laboratory, University College London, Holmbury St. Mary, Surrey RH5 6NT, UK DOI: 10.21105/joss.02801