Technological advances in high performance computing and maturing physical models allow scientists to simulate weather and climate evolutions with an increasing accuracy. While this improved accuracy allows us to explore complex dynamical interactions within such physical systems, inconceivable a few years ago, it also results in grand challenges regarding the data visualization and analytics process. We present STRIELAD, a scalable weather analytics toolkit, which allows for interactive exploration and real-time visualization of such large scale datasets. It combines parallel and distributed feature extraction using high-performance computing resources with smart level-of-detail rendering methods to assure interactivity during the complete analysis process.

In this paper, we present a software architecture and framework developed over the past decade to enable scalable and highly interactive visualizations for large datasets and display sizes. The framework integrates distributed data processing, data streaming, and dynamic scheduling to allow for view-dependent feature extraction and progressive data streaming. Additionally, the framework has been designed to support visualizations from local desktop workstations to large multi-display virtual environments.

Interactive urgent computing is a small but growing user of supercomputing resources. However there are numerous technical challenges that must be overcome to make supercomputers fully suited to the wide range of urgent workloads which could benefit from the computational power delivered by such instruments. An important question is how to connect the different components of an urgent workload; namely the users, the simulation codes, and external data sources, together in a structured and accessible manner. In this paper we explore the role of workflows from both the perspective of marshalling and control of urgent workloads, and at the individual HPC machine level. Ultimately requiring two workflow systems, by using a space weather prediction urgent use-cases, we explore the benefit that these two workflow systems provide especially when one exploits the flexibility enabled by them interoperating.

Natural disasters and epidemics are unfortunate recurring events that lead to huge societal and economic loss. Recent advances in supercomputing can facilitate simulations of such scenarios in (or even ahead of) real-time, therefore supporting the design of adequate responses by public authorities. By incorporating high-velocity data from sensors and modern high-performance computing systems, ensembles of simulations and advanced analysis enable urgent decision-makers to better monitor the disaster and to employ necessary actions (e.g., to evacuate populated areas) for mitigating these events. Unfortunately, frameworks to support such versatile and complex workflows for urgent decision-making are only rarely available and often lack in functionalities. This paper gives an overview of the VESTEC project and framework, which unifies orchestration, simulation, in-situ data analysis, and visualization of natural disasters that can be driven by external sensor data or interactive intervention by the user. We show how different components interact and work together in VESTEC and describe implementation details. To disseminate our experience three different types of disasters are evaluated: a Wildfire in La Jonquera (Spain), a Mosquito-Borne disease in two regions of Italy, and the magnetic reconnection in the Earth magnetosphere.

Large-scale numerical simulations of planetary interiors require dedicated visualization algorithms that are able to efficiently extract a large amount of information in an interactive and user-friendly way. Here we present a software framework for the visualization of mantle convection data. This framework combines real-time volume rendering, pathline visualization, and parallel coordinates to explore the fluid dynamics in an interactive way and to identify correlations between various output variables.

Over the last few years, the amount of large and complex data in the public domain has increased enormously and new challenges arose in the representation, analysis and visualization of such data. Considering the number of space missions that provided and will provide remote sensing data, there is still the need of a system that can be dispatched in several remote repositories and being accessible from a single client of commodity hardware. To tackle this challenge, at the DLR Institute for Software Technology we have defined a dual backend frontend system, enabling the interactive analysis and visualization of large-scale remote sensing data. The basis for all visualization and interaction approaches is CosmoScout VR, a visualization tool developed internally at DLR, and publicly available on Github, that allows the visualization of complex planetary data and large simulation data in real-time. The dual component of this system is based on an MPI framework, called Viracocha, that enables the analysis of large data remotely, and allows the efficient network usage about sending compact and partial results for interactive visualization in CosmoScout as soon as they are computed. A node-based interface is defined within the visualization tool, and this lets a domain expert to easily define customized pipelines for processing and visualizing the remote data. Each “node” of this interface is either linked with a feature extraction module, defined in Viracocha, or to a rendering module defined directly in CosmoScout. Being this interface completely customizable by a user, multiple pipelines can be defined over the same dataset to enhance even more the visualization feedback for analysis purposes. Being an ongoing project, on top of these tools, as a novel strategy in EO data processing and visualization, we plan to define and implement strategies based on Topological Data Analysis (TDA). TDA is an emerging set of technique for processing the data considering its topological features. These include both the geometric information associated to a point, as well all the non-geometric scalar values, like temperature and pressure, to name a few, that can be captured during a monitoring mission. One of the major theories behind TDA is Discrete Morse Theory, that, given a scalar value, is used to define a gradient on such function, extract the critical points, identify the region-of-influence of each critical point, and so on. This strategy is parameter free and enables a domain scientist to process large datasets without a prior knowledge of it. An interesting research question, that it will be investigated during this project is the correlation of changes of critical points at different time steps, and the identification of deformation (or changes) across time in the original dataset.

It is estimated that around 80% of the world’s population live in areas susceptible to at-least one major vector borne disease, and approximately 20% of global communicable diseases are spread by mosquitoes. Furthermore, the outbreaks of such diseases are becoming more common and widespread, with much of this driven in recent years by socio-demographic and climatic factors. These trends are causing significant worry to global health organisations, including the CDC and WHO, and-so an important question is the role that technology can play in addressing them. In this work we describe the integration of an epidemiology model, which simulates the spread of mosquito-borne diseases, with the VESTEC urgent computing ecosystem. The intention of this work is to empower human health professionals to exploit this model and more easily explore the progression of mosquito-borne diseases. Traditionally in the domain of the few research scientists, by leveraging state of the art visualisation and analytics techniques, all supported by running the computational workloads on HPC machines in a seamless fashion, we demonstrate the significant advantages that such an integration can provide. Furthermore we demonstrate the benefits of using an ecosystem such as VESTEC, which provides a framework for urgent computing, in supporting the easy adoption of these technologies by the epidemiologists and disaster response professionals more widely.

We introduce Virtual Planet , an application which enables researchers to interactively explore huge planetary data sets in an intuitive way. The application allows users to navigate seamlessly between planets and provides different tools and interactive visualization to analyze the data.

While technological advances in high performance computing allow for an ever-increasing accuracy in climate and weather simulations, they also lead to grand challenges regarding the data visualization and analytics process. We present a visualization framework, which allows for interactive exploration and real-time visualization of such large scale datasets in virtual reality. It combines parallel and distributed feature extraction using high-performance computing resources with octree-based level-of-detail rendering methods to assure high frame rates during the complete analysis process. When parameters such as an iso-value or the current time-step are modified, the visualization is updated progressively in a view-dependent manner. In addition, the data is shown in relation to the geographical, planetary and celestial context: the data is shown as part of our solar system. Planets are rendered with a sophisticated level-of-detail system based on the HEALPix tessellation of spheres. HEALPix tiles have equal areas and do not suffer from singularities at poles which are a common issue with other tessellations. Geographical datasets (e.g. Satellite images, digital elevation data or vector maps) are loaded from Web-Map-Services (WMS). Example datasets for Earth include, but are not limited to, Sentinel images, Open Street Map, TanDEM-X or SRTM30. NASA's SPICE library is used to compute the position of sun, planets, moons and stars.

Many scientific applications deal with data from a multitude of different sources, e.g., measurements, imaging and simulations. Each source provides an additional perspective on the phenomenon of interest, but also comes with specific limitations, e.g. regarding accuracy, spatial and temporal availability. Effectively combining and analyzing such multimodal and partially incomplete data of limited accuracy in an integrated way is challenging. In this work, we outline an approach for an integrated analysis and visualization of the atmospheric impact of volcano eruptions. The data sets comprise observation and imaging data from satellites as well as results from numerical particle simulations. To analyze the clouds from the volcano eruption in the spatiotemporal domain we apply topological methods. We show that topology-related extremal structures of the data support clustering and comparison. We further discuss the robustness of those methods with respect to different properties of the data and different parameter setups. Finally we outline open challenges for the effective integrated visualization using topological methods.

Real-time rendering of high precision shadows using digital terrain models as input data is a challenging task. Especially when interactivity is targeted and level of detail data structures are utilized to tackle huge amount of data. In this paper, we present a real-time rendering approach for the computation of hard shadows using large scale digital terrain data obtained by satellite imagery. Our approach is based on an extended horizon mapping algorithm that avoids costly pre-computations and ensures high accuracy. This algorithm is further developed to handle large data. The proposed algorithms take the surface curvature of the large spherical bodies into account during the computation. The performance issues are discussed and the results are presented. The generated images can be exploited in 3D research and aerospace related areas.

The three year European research project CROSS DRIVE (Collaborative Rover Operations and Planetary Science Analysis System based on Distributed Remote and Interactive Virtual Environments) started ...

The development of space systems involves complex interdisciplinary systems engineering. The concurrent engineering (CE) approach has been successfully applied to the early design phase of space missions. To bridge the gap between the development phases and between the different domain experts, a model based system engineering (MBSE) approach is showing promising results. To support CE and MBSE during space mission development, the German Aerospace Center (DLR) has started developing a new tool called Virtual Satellite. It offers extended software support required by CE for inter-domain communication, data exchange, dependency analysis, on the fly data analysis, data consistency while maintaining a common system model based on the MBSE approach. However, the general issues of inter-domain communication and understanding still exist and may lead to misinterpretation. To overcome this problem it is intended to take advantage of interactive 3D visualization and Virtual Reality techniques to visualize the complex system model and, thus, provide a common understanding of the system model and the intrinsic domain knowledge. Furthermore, this will promote the experts to communicate their ideas and improve the visibility of potential design issues. The paper describes the efforts taken at DLR in this direction, architecture details and advantages of adopting these techniques into space mission development from the early design phase.

The ever-increasing compute capacity of high-performance systems enables scientists to simulate physical phenomena with a high spatial and temporal accuracy. Thus, the simulation output can yield dataset sizes of many terabytes. An efficient analysis and visualization process becomes very difficult especially for explorative scenarios where users continuously change input parameters. Using a distributed rendering pipeline may relieve the visualization frontend considerably but is often not sufficient. Therefore, we additionally propose a progressive data streaming and rendering approach. The main contribution of our method is the importance-guided order of data processing for block structured datasets. This requires a dynamic scheduling of data chunks on the parallel post-processing system which has been implemented by using an R-Tree. In this paper, we demonstrate the efficiency of our implementation for view-dependent feature extraction with varying viewpoints.

With the increasing power of the HPC hardware systems, numerical simulations are heading towards exa-scale computing. Early inspection and analysis of on-going large simulations enables domain experts to obtain first insight into their running simulation process and intermediate results. Compared to conventional post-processing, such in-situ processing has the advantage of keeping data in memory, avoiding to store the large amount of raw data to disk, providing on-the-fly analysis, and preventing early failures in the simulation process. In this poster we present a distributed and scalable software infrastructure, which provides distributed in-situ data processing, feature extraction and interactive exploration at user's front-end. We have integrated and extended our system to multiple simulation applications, ranging from Lattice-Boltzmann blood flow simulation to grid based simulation for propulsion systems. A user-interactive front-end is integrated to our system, allowing to directly interact with the visualization of running simulations, gain insight, and make decisions.

With the ever-increasing capacity of high performance computing (HPC) systems, the computational simulation models become still finer and more accurate. However, the size and complexity of the data produced poses tremendous challenges for the visualization and analysis task. Especially when explorative approaches are demanded, distributed and parallel post-processing architectures have to be developed in order to allow interactive human-computer interfaces. Such infrastructures can also be exploited for the evaluation of ongoing simulation runs. The application here ranges from online monitoring to computational steering. But also remote and parallel rendering can be integrated into the overall setup. This chapter gives an overview of current solutions and ongoing research activities in this domain.

The ever increasing compute capacity of high performance computing (HPC) systems enables scientists to simulate and explore physical phenomena with an enormous spatial and temporal accuracy. On the other hand, this accuracy leads to datasets of many terabytes, petabytes, and even exabytes envisioning the up- coming exascale area projected for 2018. To understand complex physical coherences behind such a simulation, an efficient analysis and visualization is essential but also difficult, since the challenges concern all stages of the visual- ization pipeline. With this presentation we set the focus on distributed and hybrid rendering.