This study presents scaling results and a performance analysis across different supercomputers and compilers for the Met Office weather and climate model, LFRic. The model is shown to scale to large numbers of nodes which meets the design criteria, that of exploitation of parallelism to achieve good scaling. The model is written in a Domain-Specific Language, embedded in modern Fortran and uses a Domain-Specific Compiler, PSyclone, to generate the parallel code. The performance analysis shows the effect of choice of algorithm, such as redundant computation and scaling with OpenMP threads. The analysis can be used to motivate a discussion of future work to improve the OpenMP performance of other parts of the code. Finally, an analysis of the performance tuning of the I/O server, XIOS is presented.

The diversity in processor technology used by High Performance Computing (HPC) facilities is growing, and so applications must be written in such a way that they can attain high levels of performance across a range of different CPUs, GPUs, and other accelerators. Measuring application performance across this wide range of platforms becomes crucial, but there are significant challenges to do this rigorously, in a time efficient way, whilst assuring results are scientifically meaningful, reproducible, and actionable. This paper presents a methodology for measuring and analysing the performance portability of a parallel application and shares a software framework which combines and extends adopted technologies to provide a usable benchmarking tool. We demonstrate the flexibility and effectiveness of the methodology and benchmarking framework by showcasing a variety of benchmarking case studies which utilise a stable of supercomputing resources at a national scale.

The cloud microphysics scheme, CASIM, and the radiation scheme, SOCRATES, are two computationally intensive parts within the Met Office's Unified Model (UM). This study enables CASIM and SOCRATES to use accelerated multi-core systems for optimal computational performance of the UM. Using profiling to guide our efforts, we refactored the code for optimal threading and kernel arrangement and implemented OpenACC directives manually or through the CLAW source-to-source translator. Initial porting results achieved 10.02x and 9.25x speedup in CASIM and SOCRATES respectively on 1 GPU compared with 1 CPU core. A granular performance analysis of the strategy and bottlenecks are discussed. These improvements will enable UM to run on heterogeneous computers and a path forward for further improvements is provided.

Large Eddy Simulation is a critical modelling tool for scientists investigating atmospheric flows, turbulence and cloud microphysics. Within the UK, the principal LES model used by the atmospheric research community is the Met Office Large Eddy Model (LEM). The LEM was originally developed in the late 1980s using computational techniques and assumptions of the time, which means that the it does not scale beyond 512 cores. In this paper we present the Met Office NERC Cloud model, MONC, which is a re-write of the existing LEM. We discuss the software engineering and architectural decisions made in order to develop a flexible, extensible model which the community can easily customise for their own needs. The scalability of MONC is evaluated, along with numerous additional customisations made to further improve performance at large core counts. The result of this work is a model which delivers to the community significant new scientific modelling capability that takes advantage of the current and future generation HPC machines.

Due to the wide separation of time‐scales in geophysical fluid dynamics, semi‐implicit time integrators are commonly used in operational atmospheric forecast models. They guarantee the stable treatment of fast (acoustic and gravity) waves, while not suffering from severe restrictions on the time‐step size. To propagate the state of the atmosphere forward in time, a nonlinear equation for the prognostic variables has to be solved at every time step. Since the nonlinearity is typically weak, this is done with a small number of Newton or Picard iterations, which in turn require the efficient solution of a large system of linear equations with 𝒪(106−109) unknowns. This linear solve is often the computationally most costly part of the model. In this article an efficient linear solver for the LFRic next‐generation model currently being developed by the Met Office is described. The model uses an advanced mimetic finite element discretisation which makes the construction of efficient solvers challenging as compared to models using standard finite‐difference and finite‐volume methods. The linear solver hinges on a bespoke multigrid preconditioner of the Schur‐complement system for the pressure correction. By comparing it to Krylov subspace methods, the superior performance and robustness of the multigrid algorithm is demonstrated for standard test cases and realistic model set‐ups. In production mode, the model will have to run in parallel on hundreds of thousands of processing elements. As confirmed by numerical experiments, one particular advantage of the multigrid solver is its excellent parallel scalability due to its avoidance of expensive global reduction operations.

Large Eddy Simulation is a critical modelling tool for the investigation of atmospheric flows, turbulence and cloud microphysics. The models used by the UK atmospheric research community are homogeneous and the latest model, MONC, is designed to run on substantial HPC systems with very high CPU core counts. In order to future proof these codes it is worth investigating other technologies and architectures which might support the communities running their codes at the exa-scale. In this paper we present a hybrid version of MONC, where the most computationally intensive aspect is offloaded to the GPU while the rest of the functionality runs concurrently on the CPU. Developed using the directive driven OpenACC, we consider the suitability and maturity of this technology to modern Fortran scientific codes as well general software engineering techniques which aid this type of porting work. The performance of our hybrid model at scale is compared against the CPU version before considering other tuning options and making a comparison between the energy usage of the homo- and hetero-geneous versions. The result of this work is a promising hybrid model that shows performance benefits of our approach when the GPU has a significant computational workload which can not only be applied to the MONC model but also other weather and climate simulations in use by the community.

This paper describes LFRic: the new weather and climate modelling system being developed by the UK Met Office to replace the existing Unified Model in preparation for exascale computing in the 2020s. LFRic uses the GungHo dynamical core and runs on a semi-structured cubed-sphere mesh. The design of the supporting infrastructure follows object orientated principles to facilitate modularity and the use of external libraries where possible. In particular, a `separation of concerns' between the science code and parallel code is imposed to promote performance portability. An application called PSyclone, developed at the STFC Hartree centre, can generate the parallel code enabling deployment of a single source science code onto different machine architectures. This paper provides an overview of the scientific requirement, the design of the software infrastructure, and examples of PSyclone usage. Preliminary performance results show strong scaling and an indication that hybrid MPI/OpenMP performs better than pure MPI.

The Met Office’s weather and climate simulation code the Unified Model is used for both operational Numerical Weather Prediction and Climate modelling. The computational performance of the model running on parallel supercomputers is a key consideration. A Krylov sub-space solver is employed to solve the equations of the dynamical core of the model, known as ENDGame. These describe the evolution of the Earth’s atmosphere. Typically, 64-bit precision is used throughout weather and climate applications. This work presents a mixed-precision implementation of the solver, the beneficial effect on run-time and the impact on solver convergence. The complex interplay of errors arising from accumulated round-off in floating-point arithmetic and other numerical effects is discussed. A careful analysis is required. The mixed-precision solver is now employed in the operational forecast to satisfy run-time constraints without compromising the accuracy of the solution.

The Met Office's weather and climate simulation code the Unified Model is used for both operational Numerical Weather Prediction and Climate modelling. The computational performance of the model running on parallel supercomputers is a key consideration. A Krylov sub-space solver is employed to solve the equations of the dynamical core of the model, known as ENDGame. These describe the evolution of the Earth's atmosphere. Typically, 64-bit precision is used throughout weather and climate applications. This work presents a mixed-precision implementation of the solver, the beneficial effect on run-time and the impact on solver convergence. The complex interplay of errors arising from accumulated round-off in floating-point arithmetic and other numerical effects is discussed. A careful analysis is required, however, the mixed-precision solver is now employed in the operational forecast to satisfy run-time constraints without compromising the accuracy of the solution.

Weather and climate models are complex pieces of software which include many individual components, each of which is evolving under pressure to exploit advances in computing to enhance some combination of a range of possible improvements (higher spatio-temporal resolution, increased fidelity in terms of resolved processes, more quantification of uncertainty, etc.). However, after many years of a relatively stable computing environment with little choice in processing architecture or programming paradigm (basically X86 processors using MPI for parallelism), the existing menu of processor choices includes significant diversity, and more is on the horizon. This computational diversity, coupled with ever increasing software complexity, leads to the very real possibility that weather and climate modelling will arrive at a chasm which will separate scientific aspiration from our ability to develop and/or rapidly adapt codes to the available hardware. In this paper we review the hardware and software trends which are leading us towards this chasm, before describing current progress in addressing some of the tools which we may be able to use to bridge the chasm. This brief introduction to current tools and plans is followed by a discussion outlining the scientific requirements for quality model codes which have satisfactory performance and portability, while simultaneously supporting productive scientific evolution. We assert that the existing method of incremental model improvements employing small steps which adjust to the changing hardware environment is likely to be inadequate for crossing the chasm between aspiration and hardware at a satisfactory pace, in part because institutions cannot have all the relevant expertise in house. Instead, we outline a methodology based on large community efforts in engineering and standardisation, which will depend on identifying a taxonomy of key activities – perhaps based on existing efforts to develop domain-specific languages, identify common patterns in weather and climate codes, and develop community approaches to commonly needed tools and libraries – and then collaboratively building up those key components. Such a collaborative approach will depend on institutions, projects, and individuals adopting new interdependencies and ways of working.

The 2-Lagrange multiplier method is a domain decomposition method based on solving Robin problems on the subdomains. In this paper we discuss the parallel implementation of the 2-Lagrange multiplier method with cross points, as introduced and analyzed for general domains in Drury and Loisel (Sharp condition number estimates for the symmetric 2-Lagrange multiplier method. In: Domain Decomposition Methods in Science and Engineering XX. Lecture Notes in Computational Science and Engineering, vol. 91, pp. 255–261, 2013) and Loisel (SIAM J. Numer. Anal. 51(6):3062–3083, 2013). We present numerical experiments, performed on HECToR (High End Computing Terascale Resources), with different domains sizes and numbers of processors for a model problem on a square.

This paper proposes a quantum multiply-accumulator circuit (QMAC), which can perform the calculation on conventional integers faster than its classical counterpart. Whereas classically applying a multiply–adder (MAC) \(n\) times to \(k\) bit integers would require \(O(n \log k)\) parallel steps, the hybrid QMAC needs only \(O(n + k)\) steps for the exact result and \(O(n + \log k)\) steps for an approximate result. The proposed circuit could potentially be embedded in a conventional computer architecture as a quantum device or accelerator, enabling a wide range of applications to execute faster.

Quantum circuits which perform integer arithmetic could potentially outperform their classical counterparts. In this paper, a quantum circuit is considered which performs a specific computational pattern on classically represented integers to accelerate the computation. Such a hybrid circuit could be embedded in a conventional computer architecture as a quantum device or accelerator. In particular, a quantum multiply-add circuit (QMAC) using a Quantum Fourier Transform (QFT) is proposed which can perform the calculation on conventional integers faster than its conventional counterpart. Whereas classically applying a multiply-adder (MAC) n times to k bit integers would require O(n log k) parallel steps, the hybrid QMAC needs only O(n + k) steps for the exact result and O(n + log k) steps for an approximate result.

The GAP (Groups, Algebra and Programming) software is an interpreted programming language for symbolic algebra computation. It also provides a library of mathematical functionality. A key computational pattern for the GAP community is the orbit problem, that of a group acting upon a set. Computationally this maps onto the graph discovery problem. The enumeration of very large orbits corresponds to the traversal of a graph with billions of vertices. A hash table is used to check whether a vertex has been visited before during the computation. The large memory requirements of such a computation necessitates using a distributed memory machine. Building a parallel version of GAP is the goal of the HPCGAP project. Message passing (MPI) and PGAS (UPC) are considered as the models for parallelisation. UPC has some advantages over MPI as some of the data structures anticipated in a parallel implementation of GAP can be simply constructed as shared objects in a PGAS model. Moreover, some of the communication patterns are not suited to the synchronous send and receive model of message passing. For example, in a parallel implementation of a hash table, the task or thread which computes the hash of an object, then knows the table entry and thus whether hash table access is remote or local. For MPI, the usual send - receive mechanism is compromised because the receiving rank cannot determine when, and from whom a message is to be passed. One-side MPI communications can be used to circumvent the problem. Windows of remote access memory are created, and guarded by locks. In UPC, the natural, shared arrays are used, again guarded by locks, However, the locking strategy for MPI and UPC is different. In this paper, the per

We present the International Lattice Data Grid (ILDG), a loosely federated grid-of-grids for sharing data from Lattice Quantum Chromodynamics (LQCD) simulations. The ILDG comprises of metadata, file-format and web-service standards, which can be used to wrap regional data-grid interfaces, allowing seamless access to catalogues and data in a diverse set of collaborating regional grids. We discuss the technological underpinnings of the ILDG, primarily the metadata and the middleware, and offer a critique of its various aspects with the hindsight of the design work and the two years of production.

We present physical results obtained from simulations using 2+1 flavors of domain wall quarks and the Iwasaki gauge action at two values of the lattice spacing a, (a −1 = 1.73 (3) GeV and a −1 = 2.28 (3) GeV). On the coarser lattice, with 24 3 × 64× 16 points (where the 16 corresponds to Ls, the extent of the 5 th dimension inherent in the domain wall fermion (DWF) formulation

Using partially twisted boundary conditions we compute the K->pi semi-leptonic form factors in the range of momentum transfers 0 <~ q^2 <= q^2_{max}=(mK-mpi)^2 in lattice QCD with N_f=2+1 dynamical flavours. In this way we are able to determine f+(0) without any interpolation in the momentum transfer, thus eliminating one source of systematic error. This study confirms our earlier phenomenological ansatz for the strange quark mass dependence of the scalar form factor. We identify and estimate potentially significant NNLO effects in the chiral expansion that guides the extrapolation of the data to the physical point. Our main result is f+(0) = 0.9599(34)(^{+31}_{-43})(14)$, where the first error is statistical, the second error is due to the uncertainties in the chiral extrapolation of the lattice data and the last error is an estimate of potential discretisation effects.

In the beginning there was the internet, then came the world wide web, and now there is the grid. In the future perhaps there will be the cloud. In the age of persistent, pervasive, and pandemic networks I review how the lattice QCD community embraced the open source paradigm for both code and data whilst adopting the emerging grid technologies, and why having your data persistently accessible via standardized protocols and services might be a good idea.