Scientific computing is undergoing rapid transformation as advances in artificial intelligence, heterogeneous computing, automation, and data-intensive research reshape not only computational tools but also the institutions, workforce models, and collaborative practices that support scientific discovery. This report synthesizes insights from the 2026 Workshop on Next-Generation Ecosystems for Scientific Computing, the second in a three-year series focused on strengthening scientific computing ecosystems through socio-technical co-design. Workshop discussions identified four interdependent strategic themes: software ecosystems for AI-enabled scientific discovery; trust, validation, and traceability; human-AI teaming and paradigm shifts; and workforce, pedagogy, and governance. The report translates these themes into eight priorities for community action spanning shared research infrastructure, trust and traceability, user experience, human-AI teaming, workforce development, cross-sector coordination, stewardship and sustainability, and evaluation of scientific value. Together, these priorities outline directions for building scientific computing ecosystems that remain trustworthy, sustainable, innovative, and resilient as AI assumes a growing role in scientific work.

The Parameterized Task Graph (PTG) interface in the PaRSEC runtime system provides a powerful mechanism to express scalable, parameterized task dependencies for distributed heterogeneous computing. However, as applications become increasingly complex and hierarchical, developers often need to compose or merge multiple JDF (Job Data Flow) descriptions—each representing a subgraph—into a single coherent PTG, a process that is currently manual, error-prone, and difficult to generalize. In this work, we introduce a Python- and AI-assisted tool to automate the merge-to-JDF process in PaRSEC. The tool recursively analyzes two or more JDFs, identifies overlapping dataflows, and merges their task and data definitions into a unified PTG while preserving correctness and dependency semantics. We demonstrate this approach on representative PTG applications, including multi-stage solvers and AI model, like Transformer, showing that the automated merge tool substantially reduces development effort while maintaining runtime efficiency.

Template Task Graph TTG) is a multiparadigm programming model built from task flowgraphs. Each task in TTG produces data for a single consumer (send) or multiple consumers (broadcast), possibly along multiple flowgraph edges. Until now identifiers of broadcast consumers had to be encoded as explicit dynamically-computed sequences, which is crucial for composing data-dependent algorithms. In this work, we split the task discovery from the main task body and introduce the use of generators available since C++23 to generate the successor list. This improves composability by allowing us to implement multiple broadcast strategies using the on-demand key generation.

Massively parallel and heterogeneous architectures can efficiently be exploited using task-based systems. They are used for ever longer-running problems, and on ever larger supercomputers. This leads to a heightened risk of failure, making fault-tolerance capabilities for such systems a desirable addition. To this end, StarPU has started including checkpoint-restart mechanisms directly inside the runtime system. But to tolerate actual fail-stop failures, the runtime system must use a distributed-memory communication paradigm that enables the writing of fault-tolerant applications. The MPI Forum has recently started including the User-Level Failure Mitigation (ULFM) extension in the MPI Standard precisely to achieve this goal. ULFM defines a set of routines that help support operations after failures, and the behavior of MPI when failures occur. The communication engine and the task-based runtime system must thus be adapted to this specification. In particular, in order to recover the full communication capabilities, it becomes necessary to dynamically replace MPI communicators(which are distributed MPI objects that are required by most MPI routines), that include failed processes, by renewed communicators without the failed processes. To this end, in this work we introduce abstract communicators, a mechanism to replace MPI communicators during execution, including failed communicators. This enables the application to continue submission without changing its communicators, even when a failure occurs. The runtime system re-orders processes without the application noticing. We then present implementation challenges raised by this mechanism, notably on the process of handling requests on old communicators, and switching to replacements while still accepting new requests submitted by the application on its original communicators. We validate our approach with a working implementation in StarPU with neglectible overhead.

Block-sparse matrix operations are a special case of general sparse algebra where the matrix is sparsely populated with dense blocks, e.g., in sparse tensor algebra for quantum chemistry. One of the challenges of implementing distributed matrix multiplication C = A× B in general is the management of communication flows since both input matrices A and B are readily available and must be distributed to the processes computing the relevant blocks of C. In this extended article, we propose two additions to the Template Task Graph (TTG) data flow programming model that allows applications to constrain the execution of tasks and to relax the strict order of tasks that results from tasks forwarding data to successor tasks in the TTG model. We show that constraint can be used in a pure dataflow model to replace artificial control flow with a more structured approach and that mutexflows can be used to allow reordering of tasks operating on the same data, while ensuring mutual exclusion. In the context of sparse matrix multiplication, we found that the combination of constraints and mutexflows allow us to limit the number of concurrent communications and thus avoid creating a bottleneck in the network while improving communication/computation overlap through reordering of ready tasks.

Block-sparse matrix operations are a special case of general sparse algebra where the matrix is sparsely populated with dense blocks, e.g., in sparse tensor algebra for quantum chemistry. One of the challenges of implementing distributed matrix multiplication C = A× B in general is the management of communication flows since both input matrices A and B are readily available and must be distributed to the processes computing the relevant blocks of C. In this paper, we propose an addition to the Template Task Graph programming model that allows applications to constrain the execution of tasks using a flexible API. We show that such constraints can be used in a pure dataflow model to replace artificial control flow with a more structured approach. In the context of sparse matrix multiplication, we found that constraints allow us to limit the number of concurrent communications and thus avoid creating a bottleneck in the network.

This work studies an iterative algorithm running on an errorprone platform, where silent errors strike each iteration with some probability. A detector verifies correctness before taking a checkpoint but may fail to detect errors. Specifically, an error at iteration.I is detected only after iteration (I - 1) + X, where.X follows a bounded probability distribution like a truncated geometric distribution. Intuitively, the error silently amplifies during some iterations before it can be detected at distance X or higher, and there is the risk of missing an error that has struck recently but cannot be detected yet..X is bounded by D, the maximum detection latency. To mitigate undetected errors during verification, a simple strategy keeps two checkpoints and divides the execution into D - 1 iteration segments, each followed by verification and checkpoint. In steady state: (i) if verification succeeds, the oldest checkpoint is erased and replaced; (ii) if it fails, rollback occurs to the oldest verified checkpoint. This work explores whether this scheme outperforms replication and determines the optimal number of checkpoints and segment lengths, both theoretically and via Monte Carlo simulations.

This work investigates how to protect numerical iterative algorithms from all types of errors that can strike at scale: fail-stop errors (a.k.a. failures) and silent errors, striking both as computation errors and memory bit-flips. We combine various techniques: detectors for computation errors, checksums for memory errors, and checkpoint/restart for failures. The objective is to minimize the expected time per iteration of the algorithm. We design a hierarchical pattern that combines and interleaves all these fault-tolerance mechanisms, and we determine the optimal periodic pattern that achieves this objective. We instantiate these results for the performance analysis of the Preconditioned Conjugate Gradient (PCG) algorithm: we report several scenarios where the optimal pattern dramatically decreases the overhead due to error mitigation.

Given GPU accelerators’ high arithmetic capacity, reducing data motion and optimizing locality are critical to achieving high performance. The task-based programming paradigm, as employed in the PaRSEC micro-task runtime system, enables the decoupling of data distribution and computation mapping to resources from the algorithm’s base expression. In this paper, we leverage this capability to explore the performance impact of several data placement strategies–some automatic and runtime-directed, and some user-directed–for the owner-compute scheduling model in the context of split-memory accelerators. We implement three different strategies for data and task mapping: a randomized first-touch policy that assigns data randomly to an accelerator, a load-balancing strategy that assigns data to the accelerator with the lowest load, and we compare it to a user-directed strategy that minimizes cross-accelerator traffic by placing tasks according to a cross-memory bandwidth minimizing strategy. We carry the evaluation on a variety of multi-GPU accelerated systems , including the Frontier system, and demonstrate that runtime-directed automatic data placement can improve locality compared to naive strategies, but also highlight that the capability of easily having modifiable user-directed data placement is of crucial importance to achieve peak performance.

This paper highlights the most significant enhancements made to PaRSEC, a scalable task-based runtime system designed for hybrid machines, during the Exascale Computing Project (ECP). The enhancements focus on expanding the capabilities of PaRSEC to address the evolving landscape of parallel computing. Notable achievements include the integration of support for three major types of accelerators (NVIDIA, AMD, and Intel GPUs), the refinement and increased flexibility of the communication subsystem, and the introduction of new programming interfaces tailored for irregular applications. Additionally, the project resulted in the development of powerful debugging and performance analysis tools aimed at assisting users in understanding and optimizing their applications. We present a comprehensive demonstration of these advancements through a series of benchmarks and applications within ECP and beyond, thereby showcasing the enhanced capabilities of PaRSEC across the diverse architectures within the ECP, providing valuable insights into the runtime system’s adaptability and performance across varied computing environments.

Consider a High-Performance Computing (HPC) application executing on a large-scale failure-prone platform. The Fixed-Work Checkpointing (FWC) problem consists in minimizing the expected time to execute a fixed amount of work (namely a fraction or the totality of the application). Strategies for the FWC problem have received considerable attention and are well-understood. On the contrary, the dual problem, namely the Fixed-Time Checkpointing (FTC) problem, has been considered only very recently. The FTC problem consists in maximizing the expected work achieved during a fixed amount of time (namely the duration of a reservation granted to the application). This work provides a comparative overview of both problems. First we review existing strategies for the FWC problem and extend them to stochastic checkpoints, i.e., when the checkpoint is no longer a deterministic constant but obeys some probability distribution law instead. Then we provide a comprehensive study of the FTC problem. The problem turns out to be surprisingly difficult, even when restricting to taking one or two checkpoints. We provide a threshold-based heuristic to solve the general instance of the problem with an arbitrary number of checkpoints, and we have to resort to time discretization to provide an optimal strategy. We further extend this latter strategy to stochastic checkpoints.

Today, multi-GPU computing nodes are the mainstay of most high-performance computing systems. Despite significant progress in programmability, building an application that efficiently utilizes all the GPUs in a computing node is still a significant challenge, especially using the existing shared-memory and message-passing paradigms. In this aspect, the task-based dataflow programming model has emerged as an alternative for multi-GPU computing nodes.Most task-based dataflow runtimes have dynamic task mapping, where tasks are mapped to different GPUs based on the current load, but once the mapping has been established, there is no rebalancing of tasks even if an imbalance is detected. In this paper, we examine how automatic dynamic work sharing between GPUs within a compute node can improve the performance of an application through better workload distribution. We demonstrate performance improvement through dynamic work sharing using a Block-Sparse GEneral Matrix Multiplication (BSpGEMM) benchmark. Although we use PaRSEC, a task-based dataflow runtime, as the vehicle for this research, the ideas discussed here are transferable to any task-based dataflow runtime.

This work revisits I/O bandwidth -sharing strategies for HPC applications. When several applications post concurrent I/O operations, well-known approaches include serializing these operations (FCFS) or fair -sharing the bandwidth across them (FAIRSHARE). Another recent approach, I/O -Sets, assigns priorities to the applications, which are classified into different sets based upon the average length of their iterations. We introduce several new bandwidth -sharing strategies, some of them simple greedy algorithms, and some of them more complicated to implement, and we compare them with existing ones. Our new strategies do not rely on any a -priori knowledge of the behavior of the applications, such as the length of work phases, the volume of I/O operations, or some expected periodicity. We introduce a rigorous framework, namely steady-state windows, which enables to derive bounds on the competitive ratio of all bandwidth -sharing strategies for three different objectives: minimum yield, platform utilization, and global efficiency. To the best of our knowledge, this work is the first to provide a quantitative assessment of the online competitiveness of any bandwidth -sharing strategy. This theoryoriented assessment is complemented by a comprehensive set of simulations, based upon both synthetic and realistic traces. The main conclusion is that two of our simple and low -complexity greedy strategies significantly outperform FCFS, FAIRSHARE and I/O -Sets, and we recommend that the I/O community would implement them for further assessment.

The Young/Daly formula provides an approximation of the optimal checkpoint period for a parallel application executing on a supercomputing platform. The Young/Daly formula was originally designed for preemptible tightly-coupled applications. We provide some background and survey various application scenarios to assess the usefulness and limitations of the formula.

Task-based runtime systems, characterized by their dynamic execution models and optimized resource management, contribute significantly to the computational revolution. They enable the development of more intricate and adaptable algorithms, essential in the field of computational science. This paper provides an in-depth exploration of the PaRSEC task-based runtime system, particularly focusing on its versatility in managing a variety of matrix computations. More specifically, we examine PaRSEC’s role in enhancing efficiency when solving linear systems and processing dense, low-rank, mixed-precision, and sparse matrix operations, which are crucial in scientific applications, e.g., climate/weather prediction and 3D unstructured mesh deformation-the primary focus of this study. Through experimentation and analysis, we showcase PaRSEC’s ability to significantly boost computational efficiency and scalability across a range of computationally intensive and less intensive tasks on various hardware architectures. Our findings not only underscore the potential of PaRSEC in advancing sustainable, efficient, and accurate domain modeling and simulation but also emphasize the growing necessity of task-based runtime systems in supporting the next generation of matrix computations.

This work considers an application executing for a fixed duration, namely the length of the reservation that it has been granted. The checkpoint duration is a stochastic random variable that obeys some well-known probability distribution law. The question is when to take a checkpoint towards the end of the execution, so that the expectation of the work done is maximized. We address two scenarios. In the first scenario, a checkpoint can be taken at any time; despite its simplicity, this natural problem has not been considered yet (to the best of our knowledge). We provide the optimal solution for a variety of probability distribution laws modeling checkpoint duration. The second scenario is more involved: the application is a linear workflow consisting of a chain of tasks with IID stochastic execution times, and a checkpoint can be taken only at the end of a task. First, we introduce a static strategy where we compute the optimal number of tasks before the application checkpoints at the beginning of the execution. Then, we design a dynamic strategy that decides whether to checkpoint or to continue executing at the end of each task. We instantiate this second scenario with several examples of probability distribution laws for task durations.

This paper revisits distributed termination detection algorithms in the context of High-Performance Computing (HPC) applications. We introduce an efficient variant of the Credit Distribution Algorithm ( CDA ) and compare it to the original algorithm ( HCDA ) as well as to its two primary competitors: the Four Counters algorithm ( 4C ) and the Efficient Delay-Optimal Distributed algorithm ( EDOD ). We analyze the behavior of each algorithm for some simplified task-based kernels and show the superiority of CDA in terms of the number of control messages. We then compare the implementation of these algorithms over a task-based runtime system, PaRSEC and show the advantages and limitations of each approach in a real implementation.

This paper revisits distributed termination detection algorithms in the context of High-Performance Computing (HPC) applications. We introduce an efficient variant of the Credit Distribution Algorithm ( CDA ) and compare it to the original algorithm ( HCDA ) as well as to its two primary competitors: the Four Counters algorithm ( 4C ) and the Efficient Delay-Optimal Distributed algorithm ( EDOD ). We analyze the behavior of each algorithm for some simplified task-based kernels and show the superiority of CDA in terms of the number of control messages. We then compare the implementation of these algorithms over a task-based runtime system, PaRSEC and show the advantages and limitations of each approach on a practical implementation

In this paper, we explore the composition capabilities of the Template Task Graph (TTG) programming model. We show how fine-grain composition of tasks is possible in TTG between DAGs belonging to different libraries, even in a distributed setup. We illustrate the benefits of this fine-grain composition on a linear algebra operation, the matrix inversion via the Cholesky method, which consists of three operations that need to be applied in sequence.Evaluation on a cluster of many core shows that the transparent fine-grain composition implements the complex operation without introducing unnecessary synchronizations, increasing the overlap of communication and computation, and thus improving significantly the performance of the entire composed operation.