This deliverable report focuses on the final stochastic optimization results obtained within the EXAscale Quantification of Uncertainties for Technology and Science Simulation (ExaQUte) project. Details on a novel wind inlet generator that is able to incorporate local wind-field data through a deep-learned rapid distortion model and generates the turbulent wind data during run-time is presented in section 2. Section 3 presents the results of the overall stochastic optimization procedure applied to a twisted tapered tower with multiple design parameters within an uncertain synthetic wind field. Thereby, the significance of the developed methods and the obtained results are discussed and their integration in industrial wind-engineering workflows is outlined in section 4.

In Deliverable D8.1 (M18) we presented a first version of the Dissemination Plan for the ExaQUte project. The present document, prepared during the 2nd (and last) review period of the project, represents the updated version of the Dissemination Plan of ExaQUte, and therefore builds on the aforementioned deliverable. This document, thus, focuses on the new activities that have been undertaken from M18 to M42 regarding the dissemination activities. It should be mentioned that the pandemic situation that started in Mach 2020 (actually, in Spain we were sent to confine to our houses the day after the celebration of the First Review meeting of this project) has had an impact in our life, and in our project, particularly in the dissemination actions. The lockdown made impossible most of the activities related to dissemination (travels, conferences, consortium meetings…). But we reinvented our work and found different ways to do things and fulfil pour obligations. Still, in this deliverable, you will find indeed a difference between the type of action that we carried on up to M18 and our activities after M18.

This report presents the latest methods of optimisation under uncertainties investigated in the ExaQUte project, and their applications to problems related to civil and wind engineering. The measure of risk throughout the report is the conditional value at risk. First, the reference method is presented: the derivation of sensitivities of the risk measure; their accurate computation; and lastly, a practical optimisation algorithm with adaptive statistical estimation. Second, this method is directly applied to a nonlinear relaxation oscillator (FitzHugh–Nagumo model) with numerical experiments to demonstrate its performance. Third, the optimisation method is adapted to the shape optimisation of an airfoil and illustrated by a large-scale experiment on a computing cluster. Finally, the benchmark of the shape optimisation of a tall building under a turbulent flow is presented, followed by an adaptation of the optimisation method. All numerical experiments showcase the open-source software stack of the ExaQUte project for large-scale computing in a distributed environment.

In this work we focus on reducing the wall clock time required to compute statistical estimators of highly chaotic incompressible flows on high performance computing systems. Our approach consists of replacing a single long-term simulation by an ensemble of multiple independent realizations, which are run in parallel with different initial conditions. A failure probability convergence criteria must be satisfied by the statistical estimator of interest to assess convergence. Its error analysis leads to the identification of two error contributions: the initialization bias and the statistical error. We propose an approach to systematically detect the burn-in time in order to minimize the initialization bias, accompanied by strategies to reduce simulation cost. The framework is validated on two very high Reynolds number obstacle problems of wind engineering interest in a high performance computing environment.

We study the use of multi-level Monte Carlo methods for wind engineering. This report brings together methodological research on uncertainty quantification and work on target applications of the ExaQUte project in wind and civil engineering. First, a multi-level Monte Carlo for the estimation of the conditional value at risk and an adaptive algorithm are presented. Their reliability and performance are shown on the time-average of a non-linear oscillator and on the lift coefficient of an airfoil, with both preset and adaptively refined meshes. Then, we propose an adaptive multi-fidelity Monte Carlo algorithm for turbulent fluid flows where multilevel Monte Carlo methods were found to be inefficient. Its efficiency is studied and demonstrated on the benchmark problem of quantifying the uncertainty on the drag force of a tall building under random turbulent wind conditions. All numerical experiments showcase the open-source software stack of the ExaQUte project for large-scale computing in a distributed environment.

This deliverable presents the nal release of the ExaQUte framework as result of task 4.6 of the project focused on the framework development and optimization. The rst part of the document presents an overview of the dierent parts of the ExaQUte framework providing the links to the repositories where the code of the dierent components can be found as well as the installation and usage guidelines. These repositories will include the nal version of the ExaQUte API and its implementation for the runtimes provided in the project (PyCOMPSs/COMPSs and Quake). The second part of the document presents a performance analysis of the framework by performing strong and weak scaling experiments. In this case, we have focused on the analysis of the new features introduced during the last part of the project to support and optimize the execution of MPI solvers inside the framework. The support for OpenMP was already reported in Deliverable D4.3 [21]. The results of the experiments demonstrate that the proposed framework allow to reach very good scalability for the analysed Monte Carlo problems.

This deliverable focuses on the implementation of deterministic optimization algorithms and problem solvers within KRATOS open-source software. One of the main challenges of optimization algorithms in Finite-Element based optimization is how to get the gradient of response functions which are used as objective and constraints when this is not available in an explicit form. The idea is to use local sensitivity analysis to get the gradient of the response function(s)

In this report, we study the use of Multi-Level Monte Carlo (MLMC) methods for time dependent problems. It was found that the usability of MLMC methods depends strongly on whether or not the underlying time dependent problem is chaotic in nature. Numerical experiments are conducted on both simple problems, as well as fluid flow problems of practical interest to the ExaQUte project, to demonstrate this. For the non-chaotic cases, the hypotheses that enable the use of MLMC methods were found to be satisfied. For the chaotic cases, especially the case of high Reynolds’ number fluid flow, the hypotheses were not satisfied. However, it was found that correlations between the different levels were high enough to merit the use of multi-fidelity or control-variate approaches. It was also noted that MLMC methods could work for chaotic problems if the time window of analysis were chosen to be small enough. Future studies are proposed to examine this possibility.

This document describes the industrial application, on which the developments of the project are implemented, and the CFD set-up. The developments are implemented over six analysis cases with increasing complexity starting from a 2D geometry with mean wind inflow to a 3D geometry with turbulent inflow and real-time shape optimization. The application represents the CAARC tall building model, which has served as a benchmark model for many studies since the 1970’s when it was first developed. Base moments (bending and torsional moments) of the building are extracted for validation by comparison of the results with the benchmark study. Page 3 of 19 Deliverable 7.1

This report brings together methodological research on stochastic optimisation and work on benchmark and target applications of the ExaQute project, with a focus on unsteady problems. A practical, general method for the optimisation of the conditional value at risk is proposed. Three different optimisation problems are described: an oscillator problem selected as a suitable trial and illustration case; the shape optimisation of an airfoil, chosen as a benchmark application in the project; the shape optimisation of a tall building, which is the challenging target application set for ExaQUte. For each problem, the current developments and results are presented, the application of the proposed method is discussed, and the work to be done until the end of the project is laid out.

In the following sections, the formulation of a possible stochastic optimisation problem relevant to the ExaQUte project is first presented

This document presents a description of the octree mesh-generation capabilities and of the parallel mesh adaptation kernel. As it is discussed in Section 1.3.2 of part B of the project proposal there are two parallel research lines aimed at developing scalable adaptive mesh refinement (AMR) algorithms and implementations. The first one is based on using octree-based mesh generation and adaptation for the whole simulation in combination with unfitted finite element methods (FEMs) and the use of algebraic constraints to deal with non-conformity of spaces. On the other hand the second strategy is based on the use of an initial octree mesh that, after make it conforming through the addition of templatebased tetrahedral refinements, is adapted anisotropically during the calculation. Regarding the first strategy the following items are included:

This deliverable presents the activities performed on the ExaQUte project task 4.5 Development of interface to fast local storage. The activities have been focused in two aspects: reduction of the storage space used by applications and design and implementation of an interface that optimizes the use of fast local storage by MPI simulations involved in the project applications. In the rst case, for one of the environments involved in the project (PyCOMPSs) the default behavior is to keep all intermediate les until the end of the execution, in case these les are reused later by any additional task. In the case of the other environment (HyperLoom), all les are deleted by default. To unify these two behaviours, the calls \delete object" and \detele le"have been added to the API and a ag \keep" that can be set to true to keep the les and objects that maybe needed later on. We are reporting results on the optimization of the storage needed by a small case of the project application that reduces the storage needed from 25GB to 350MB. The second focus has been on the de nition of an interface that enables the optimization of the use of local storage disk. This optimization focuses on MPI simulations that may be executed across multiple nodes. The added annotation enables to de ne access patters of the processes in the MPI simulations, with the objective of giving hints to the runtime of where to allocate the di erent MPI processes and reduce the data transfers, as well as the storage usage.

This deliverable report focuses on the main Uncertainty Quanti cation (UQ) results obtained within the EXAscale Quanti cation of Uncertainties for Technology and Science Simulation (ExaQUte) project. Details on the turbulent wind inlet generator, that enables the supply of random, yet steady, wind velocity boundary conditions during run-time, are given in section 2. This enables the developed UQ workflow, whose results are presented on the basis of the Commonwealth Advisory Aeronautical Council (CAARC) as described in Deliverable 7.1. Finally, the completed UQ workflow and the results are evaluated from an application-driven wind engineering point of view. Thereby, the significance of the developed methods and the obtained results are discussed and their applicability in practical wind-engineering applications is tested through a complete test-run of the UQ workflow.

This deliverable presents the final software release of Kratos Multiphysics, together with the XMC library, Hyperloom and PyCOMPSs API definitions [13]. This release also contains the latest developements on MPI parallel remeshing in ParMmg. This report is meant to serve as a supplement to the public release of the software. Kratos is “a framework for building parallel, multi-disciplinary simulation software, aiming at modularity, extensibility, and high performance. Kratos is written in C++, and counts with an extensive Python interface”. XMC is “a Python library for parallel, adaptive, hierarchical Monte Carlo algorithms, aiming at reliability, modularity, extensibility and high performance“. Hyperloom and PyCOMPSs are environments for enabling parallel and distributed computation. ParMmg is an open source software which offers the parallel mesh adaptation of three dimensional volume meshes.

The main focus of this deliverable is testing and benchmarking the available infrastructure using the execution frameworks PyCOMPSs and HyperLoom. A selected benchmark employing the Multi Level Monte Carlo (MLMC) algorithm was run on two systems: TIER-0 (MareNostrum4) and TIER-1 (Salomon) supercomputers. In both systems, good performance scalability was achieved.

In this deliverable we provide the details related to the design, implementation, and scalability analysis of Space Time Balancing Domain Decomposition by Constraints (STBDDC) preconditioners that have been implemented in the FEMPAR project [8]. First, we describe the state of the art of space-time methods in Sect. 2 and we then provide some details of our particular implementation in Sect. 3. Next, in Sect. 4, we present a detailed description of the numerical experiments performed during the project showing the excellent scalability results that these algorithms permit to achieve. At the same time, we show the limitations of these algorithms when dealing with nonlinear problems. Finally we draw some concluding remarks in Sect. 5.

This deliverable focuses on the denition of a common API for the PyCOMPSs programming model and HyperLoom scheduler provided respectively by BSC and IT4I. The objective of the work is to hide the details of the actual task scheduling technology, so that the Multi Level Monte Carlo Python engine is agnostic of the backend being employed. It includes the description of: Common API for calls Examples of usage Basic Documentation The document also contains an initial description on how MPI-distributed data shall be treated from the scheduling point of view.

The ExaQUte project participates in the Pilot on Open Research Data launched by the European Commission (EC) along with the H2020 program. This pilot is part of the Open Access to Scientific Publications and Research Data program in H2020. The goal of the program is to foster access to research data generated in H2020 projects. The use of a Data anagement Plan (DMP) is required for all projects participating in the Open Research Data Pilot, in which they will specify what data will be kept for the longer term. The underpinning idea is that Horizon 2020 beneficiaries have to make their research data findable, accessible, interoperable and re-usable (FAIR), to ensure it is soundly managed.