In transit visualization offers a desirable approach to performing in situ visualization by decoupling the simulation and visualization components. This decoupling requires that the data be transferred from the simulation to the visualization, which is typically done using some form of aggregation and redistribution. As the data distribution is adjusted to match the visualization’s parallelism during redistribution, the data transport layer must have knowledge of the input data structures to partition or merge them. In this chapter, we will discuss an alternative approach suitable for quickly integrating in transit visualization into simulations without incurring significant overhead or aggregation cost. Our approach adopts an abstract view of the input simulation data and works only on regions of space owned by the simulation ranks, which are sent to visualization clients on demand.
This book provides an overview of the emerging field and demonstrates idea about the in situ visualization, an emerging processing paradigm
Cinema is a flexible in situ visualization ecosystem that combines data extracts with viewers and analysis capabilities to support in situ, post hoc and hybrid approaches for data processing. With data extracts that include metadata, images, meshes, and other data types, Cinema databases generated in situ are a central component of post hoc analysis workflows. These workflows support visualization and exploration of data, verification and validation tasks, and leverage computer vision and statistical techniques for post hoc analysis. This chapter describes the Cinema approach, its database specification, and demonstrates its use through example workflows.
In situ visualization increasingly involves rendering large numbers of images for post hoc exploration. As both the number of images to be rendered and the data being rendered are large, the scalability of the rendering component is of key concern. Furthermore, the renderer must be able to support a wide range of data distributions, simulation configurations, and HPC systems to provide the flexibility required for a portable, general purpose in situ rendering package. In this chapter, we discuss recent developments in OSPRay’s support for MPI-parallel applications to provide a flexible and scalable rendering API, with a focus on how these developments can be applied to enable scalable, high-quality in situ visualization.
The visualization community has invested decades of research and development into producing large-scale production visualization tools. Although in situ is a paradigm shift for large-scale visualization, much of the same algorithms and operations apply regardless of whether the visualization is run post hoc or in situ. Thus, there is a great benefit to taking the large-scale code originally designed for post hoc use and leveraging it for use in situ. This chapter describes two in situ libraries, Libsim and Catalyst, that are based on mature visualization tools, VisIt and ParaView, respectively. Because they are based on fully-featured visualization packages, they each provide a wealth of features. For each of these systems we outline how the simulation and visualization software are coupled, what the runtime behavior and communication between these components are, and how the underlying implementation works. We also provide use cases demonstrating the systems in action. Both of these in situ libraries, as well as the underlying products they are based on, are made freely available as open-source products. The overviews in this chapter provide a toehold to the practical application of in situ visualization.
One key challenge when doing in situ processing is the investment required to add code to numerical simulations needed to take advantage of in situ processing. Such instrumentation code is often specialized, and tailored to a specific in situ method or infrastructure. Then, if a simulation wants to use other in situ tools, each of which has its own bespoke API [4], then the simulation code team will quickly become overwhelmed with having a different set of instrumentation APIs, one per in situ tool or method. In an ideal situation, such instrumentation need happen only once, and then the instrumentation API provides access to a large diversity of tools. In this way, a data producer’s instrumentation need not be modified if the user desires to take advantage of a different set of in situ tools. The SENSEI generic in situ interface addresses this challenge, which means that SENSEI-instrumented codes enjoy the benefit of being able to use a diversity of tools at scale, tools that include Libsim, Catalyst, Ascent, as well as user-defined methods written in C++ or Python. SENSEI has been shown to scale to greater than 1M-way concurrency on HPC platforms, and provides support for a rich and diverse collection of common scientific data models. This chapter presents the key design challenges that enable tool and processing portability at scale, some performance analysis, and example science applications of the methods.
In high-performance parallel in situ processing, the term in transit processing refers to those configurations where data must move from a producer to a consumer that runs on separate resources. In the context of parallel and distributed computing on an HPC platform one of the central challenges is to determine a mapping of data from producer ranks to consumer ranks. This problem is complicated by the heterogeneity that arises in producer-consumer pairs, such as when producer and consumer codes have different levels of concurrency, different scaling characteristics, or different data models. The resulting mapping and movement of data from M producer to N consumer ranks can have a significant impact on aggregate application performance, particularly when the data consumer requires only a subset of the overall data for its task. This chapter focuses on the design considerations that underlie SENSEI’s implementation to this challenging problem. These design considerations extend the core SENSEI architecture and include ideas like the need to accommodate flexibility in the choice of different partitioning methods, the ability for a data consumer to request and receive only the subset of data needed for its particular operation, and the ability to leverage any of several different data transport tools. The idea of proximity portability, being able to use different data transport methods as part of an in transit workflow, is illustrated through the use of three different transport layers where switching from one transport tool to another is accomplished with only a configuration file change. The chapter also includes a performance analysis summary showing the performance gains that are possible in terms of multiple metrics, such as memory footprint, time to solution, and amount of data moved, when using optimized partitioners in an in transit setting, gains that are made possible by the implementation shaped by specific design considerations.
In situ workflows manage the coordination and communication in a directed graph of heterogeneous tasks executing simultaneously in an high-performance computing system. The communication through the graph can be modeled as a dataflow, and Decaf is a software library for managing the dataflow for in situ workflows. Decaf includes a Python API to define a workflow, creating a complete stand-alone system, but the dataflow design also allows Decaf to support the communication needs of other workflow management systems, because a science campaign may be composed of several workflow tools. Decaf creates efficient parallel communication channels over MPI, including arbitrary data transformations ranging from simple data forwarding to complex data redistribution. Decaf provides three building blocks: (i) a lightweight data model that enables users to define the policies needed to preserve semantic integrity during data redistribution, (ii) flow control designed to prevent overflows in the communication channels between tasks, and (iii) a data contract mechanism that allows users to specify the required data in the parallel communication of the workflow tasks. Decaf has been used in a variety of applications. Two examples are highlighted. The first case is from materials science, where the science campaign consists of several workflow tools that cooperate, and Decaf supports these tools as the dataflow layer. The second problem is motivated by computational cosmology, where the in situ workflow consists of three parallel tasks: synthetic particle generation, Voronoi tessellation, and density estimation.
This chapter describes methodologies to perform in situ computations at desired intervals along with the simulations for different execution modes. This needs to be done in a way such that the simulation throughput is minimally impacted and the analysis output is available immediately within desired intervals. We describe the formulation of optimal resource allocation for simulation and in situ analysis computations as constrained integer linear programs so that the end-to-end simulation-analysis time is minimized. In particular, we describe the scheduling of in situ analyses as a numerical optimization problem to maximize the number of online analyses and minimize overall runtime, subject to resource constraints such as I/O bandwidth, network bandwidth, rate of computation and available memory. We also demonstrate the effectiveness of our approach through real application case studies on supercomputers.
While our computational capability to generate raw data grows, the ability to store, transform, and draw conclusions from scientific data is lagging. Beginning with the introductory chapter and continuing throughout much of this book, we see numerous examples of how in situ processing can help close the gap between data generation and data analysis. This chapter expands the discussion of in situ methods beyond when and where data are processed, to how data are represented. Rethinking the way that scientific data are represented can empower subsequent visualization and analysis, especially when such data transformations are performed in situ. Scientific data may be transformed by recasting to a data model fundamentally different from the discrete pointwise or element-wise datasets produced by computational models. In Multivariate Functional Approximation, or MFA, scientific datasets are redefined in a hypervolume of piecewise-continuous basis functions. Compared with existing discrete models, the continuous functional model can save space while affording many of the same spatiotemporal analyses without reverting back to the discrete form. In this chapter, modeling the MFA, in situ, is presented. The data model and modeling approach are parallelized for high-performance computing. A lightweight and efficient method of enforcing high-degree continuity across subdomains in the parallel decomposition is also included. The MFA can subsequently be used post hoc to evaluate points and derivatives anywhere in the domain, facilitating numerous analysis and visualization applications.
This chapter describes Ascent, a production library for in situ visualization and analysis on exascale architectures. It begins by describing the library’s focal points: minimizing encumbrance on simulation codes and enabling diverse and powerful capabilities. The chapter then describes Ascent’s abstractions, interface, and design. It concludes with success stories that highlight its capabilities: in situ visualization of a 97.8 billion element inertial confinement fusion simulation using 16,384 GPUs, delivering radiography capabilities for a Kelvin-Helmholtz simulation, and native rendering of higher-order elements.
This chapter considers exploratory flow visualization of time-dependent vector fields via in situ extraction of Lagrangian representations. The Lagrangian perspective is more capable than the traditional approach of incorporating the increased spatiotemporal data afforded by in situ processing, creating significantly better trade-offs with respect to accuracy and storage. For example, in situ Lagrangian-based flow analysis has delivered the same accuracy as the traditional approach with less than 2% of the storage, or 10X greater accuracy with the same storage. The chapter begins by discussing the Lagrangian frame of reference and how this frame of reference can be used as an in situ operator for data reduction. Next, opportunities for achieving maximum information per byte—where particles are placed, how they are terminated, and how much information to store per particle trajectory are discussed. The chapter then considers post hoc exploration using the Lagrangian representation, as well as the corresponding challenges involved. Finally, the chapter concludes with a qualitative evaluation to demonstrate the efficacy of the technique and a discussion of the current state of the art.
Wavelet compression is a popular approach for reducing data size while maintaining high data integrity. This chapter considers how wavelet compression can be used for data visualization and post hoc exploration on supercomputers. There are three major parts in this chapter. The first part describes the basics of wavelet transforms, which are essential signal transformations in a wavelet compression pipeline, and how their properties can be used for data compression. The second part analyzes the efficacy of wavelet compression on scientific data, with a focus on analyses involving scientific visualizations. The third part evaluates how well wavelet compression fits in an in situ workflow on supercomputers. After reading this chapter, readers should have a high-level understanding of how wavelet compression works, as well as its efficacy for in situ compression and post hoc exploration.
This chapter complements the Preface to this book. For more discussion on how to read this book, as well as information on the book itself, its purpose, and topics covered, we refer the reader to the Preface. Instead, this chapter provides background and an overview of foundational topics for in situ visualization for computational science. Section 1 provides a historical perspective of scientific visualization on supercomputers and why computing trends are increasingly mandating in situ processing. Section 2 presents an overview of in situ system types. The chapter concludes with Sect. 3, which highlights leading challenges and solutions for in situ processing. After reading this chapter, a beginner to this field should have sufficient context to read any of the subsequent chapters in this book, in any order.
Galaxy is a multi-tenant server platform for visualization and analysis of data distributed across many nodes of a supercomputer. It presents an interface through which multiple concurrent clients can connect to perform visualization and analysis tasks on shared data. Galaxy minimizes the memory footprint of visualization and analysis tasks to reduce the impact of analysis on the simulation. It does so by utilizing a visualization renderer that incorporates common visualization techniques directly in the rendering process. The client GUI provides a data-flow programming paradigm that enables users to connect to the Galaxy Multiserver, devise complex multi-step analytics workflows, and visualize results on their desktop.
This chapter presents a study of parameter adaptation in situ, exploring the resulting trade-offs in rendering quality and workload distribution. Four different use cases are analyzed with respect to configuration changes. First, the performance impact of load balancing and resource allocation variants on both simulation and visualization is investigated using the MegaMol framework. Its loose coupling scheme and architecture enable minimally invasive in situ operation without impacting the stability of the simulation with (potentially) experimental visualization code. Second, Volumetric Depth Images (VDIs) are considered: a compact, view-dependent intermediate representation that can efficiently be generated and used for post hoc exploration. A study of their inherent trade-offs regarding size, quality, and generation time provides the basis for parameter optimization. Third, streaming for remote visualization allows a user to monitor the progress of a simulation and to steer visualization parameters. Compression settings are adapted dynamically based on predictions via convolutional neural networks across different parts of images to achieve high frame rates for high-resolution displays like powerwalls. Fourth, different performance prediction models for volume rendering address offline scenarios (like hardware acquisition planning) as well as dynamic adaptation of parameters and load balancing. Finally, the chapter concludes by summarizing overarching approaches and challenges, discussing the potential role that adaptive approaches can play in increasing the efficiency of in situ visualization.
The Adaptable I/O System (ADIOS) provides a publish/subscribe abstraction for data access and storage. The framework provides various engines for producing and consuming data through different mediums (storage, memory, network) for various application scenarios. ADIOS engines exist to write/read files on a storage system, to couple independent simulations together or to stream data from a simulation to analysis and visualization tools via the computer’s network infrastructure, and to stream experimental/observational data from the producer to data processors via the wide-area-network. Both lossy and lossless compression are supported by ADIOS to provide for seamless exchange of data between producer and consumer. In this work we provide a description for the ADIOS framework and the abstractions provided. We demonstrate the capabilities of the ADIOS framework using a number of examples, including strong coupling of simulation codes, in situ visualization running on a separate computing cluster, and streaming of experimental data between Asia and the United States.
Multi-run numerical simulations using supercomputers are increasingly used by physicists and engineers for dealing with input data and model uncertainties. Most of the time, the input parameters of a simulation are modeled as random variables, then simulations are run a (possibly large) number of times with input parameters varied according to a specific design of experiments. Uncertainty quantification for numerical simulations is a hard computational problem, currently bounded by the large size of the produced results. This book chapter is about using in situ techniques to enable large scale uncertainty quantification studies. We provide a comprehensive description of Melissa, a file avoiding, adaptive, fault-tolerant, and elastic framework that computes in transit statistical quantities of interest. Melissa currently implements the on-the-fly computation of the statistics necessary for the realization of large scale uncertainty quantification studies: moment-based statistics (mean, standard deviation, higher orders), quantiles, Sobol’ indices, and threshold exceedance.
As the era of exascale computing approaches, the need for effective, scalable, and flexible data reduction techniques is becoming more and more prominent. As discussed in the introductory chapter, this need is primarily due to the bottleneck stemming from output data size and I/O speed compared to the ever-increasing computing speed. With this chapter, we consider a promising solution: data summarization techniques that work in the in situ environment while the data is getting produced, and preserve the important information from the data compactly, which minimizes information loss and enables a variety of post hoc analyses. Specifically, this chapter shows that statistical distribution-based in situ data summaries are a pragmatic solution and able to preserve important statistical data features. Using only the in situ generated statistical data summaries, which is significantly smaller in size compared to the original raw data, a wide range of data analysis and visualization tasks can be performed such as feature detection, extraction, tracking, query-driven analysis, etc. In addition, reconstruction of the full-resolution data is also possible, in order to visualize the data in its entirety with the added advantage of uncertainty quantification. To this end, this chapter presents several distribution-based data modeling algorithms, considering both their in situ performance and the usefulness of their distribution data summaries on several application studies.
With exascale supercomputers on the horizon, data-driven in situ data reduction is a very important topic that potentially enables post hoc data visualization, reconstruction, and exploration with the goal of minimal information loss. Sophisticated sampling methods provide a fast approximation to the data that can be used as a preview to the simulation output without the need for full data reconstruction. More detailed analysis can then be performed by reconstructing the sampled data set as necessary. Other data reduction methods such as compression techniques can still be used with the sampled outputs to achieve further data reduction. Sampling can be achieved in the spatial domain (which data locations are to be stored?) and/or temporal domain (which time steps to be stored?). Given a spatial location, data-driven sampling approaches take into account its local properties (such as scalar value, local smoothness etc.) and multivariate association among scalar values to determine the importance of a location. For temporal sampling, changes in the local and global properties across time steps are taken into account as importance criteria. In this chapter, spatial sampling approaches are discussed for univariate and multivariate data sets and their use for effective in situ data reduction is demonstrated.
