Modern big data workflows are characterized by computationally intensive kernels. The simulated results are often combined with knowledge extracted from AI models to ultimately support decision-making. These energy-hungry workflows are increasingly executed in data centers with energy-efficient hardware accelerators since FPGAs are well-suited for this task due to their inherent parallelism. We present the H2020 project EVEREST, which has developed a system development kit (SDK) to simplify the creation of FPGA-accelerated kernels and manage the execution at runtime through a virtualization environment. This paper describes the main components of the EVEREST SDK and the benefits that can be achieved in our use cases.

The computational requirements of artificial intelligence workloads are growing exponentially. In addition, more and more compute is moved towards the edge due to latency or localization constraints. At the same time, Dennard scaling has ended and Moore's law is winding down. These trends created an opportunity for specialized accelerators including field-programmable gate arrays (FPGAs), but the poor support and usability of today's tools prevents FPGAs from being deployed at scale for deep neural network (DNN) inference applications. In this work, we propose an organic compiler - DOSA - that drastically lowers the barrier for deploying FPGAs. DOSA builds on the operation set architecture concept and integrates the DNN accelerator components generated by existing DNN-to-FPGA frameworks to produce an overall efficient solution. DOSA starts from DNNs represented in the community standard ONNX and automatically implements model- and data-parallelism, based on the performance targets and resource footprints provided by the user. Deploying a DNN using DOSA on 9 FPGAs exhibits a speedup of up to 52 times compared to a CPU and 18 times compared to a GPU.

In the last years, advances in genome sequencing technologies have enabled the proliferation of genomic applications that guide personalized medicine. These applications have an enormous computational cost due to the large amount of genomic data they process. The first step in many of these applications consists in aligning DNA reads against a reference genome. Very recently, the wavefront alignment (WFA) algorithm has been introduced, significantly reducing the execution time of the read alignment process. This paper presents the first FPGA-based hardware/software co-designed accelerator of such relevant algorithm. Compared to the reference WFA CPU-only implementation, the proposed accelerator achieves performance speedups of up to 13.5× while consuming up to 14.6× less energy when aligning short reads. When aligning long reads, the proposed accelerator achieves speedups of up to 9.9× while consuming up to 10.9× less energy.

Today's rapid progress in AI and science is largely fueled by the availability of ever larger and more powerful compute systems. The “classic” HPC systems targeted at executing complex workflows and simulations have recently crossed the exaflop boundary in terms of their double-precision floating point performance. At the same time, new systems targeted at training large AI-models use alternative number representations and are already pushing the limits well beyond the ten exaflop mark. To continue scaling the performance of large HPC systems, system architects need to address several barriers including the slowdown of Moore's law, energy density limitations, production yield challenges and practical limits to overall power consumption in the 10s-of- MW range. All recent #1 HPC systems are already relying on specialized, heterogenous components to offset the slowdown. As specialization continues and advances, the heterogeneity will evolve from today's CPU-GPU combinations into a broad set of more specialized accelerators, but also entirely new computing paradigms, e.g., Quantum computing are emerging. With the continued scaling of total system size and the compute density within a single node, the intra- and inter-node communication requirements increase accordingly. Today, most available interconnect fabrics that support symmetric multiprocessing (SMP) and/or asymmetric variants of cache-coherent communication are based on proprietary implementations, which prevent the assembly of heterogeneous high-performance systems from components from more than a single vendor. Hence, system architects are looking at ways assemble innovative high-performance heterogenous systems using open standards. Under continued cost constraints, better utilization is desired to match hardware configuration to software usage needs. The ability to compose virtual compute nodes from a set of disaggregated components is a natural way of approaching the problem. The first challenge of composability that is currently being tackled is memory disaggregation. A vision of higher utilization and resource sharing is appealing, but low latency and high bandwidth need to be maintained. All these trends and limitations demand a fresh look at the architectures enabling an ecosystem from which domain-specific high-performance computing systems can be assembled. In this presentation, we discuss the motivation and requirements for a new node level and rack scale architecture as well as the need for an open standards-based, composable, high-performance interconnect fabric. The architecture of this system needs to be accompanied by an open and interoperable software stack as well as a fine-grained control plane. The control plane enables and supports composability under tight security and performance constraints. While composability originated to increase the efficiency of heterogenous computer systems, more recently, it has been proposed as a means for heterogeneous components to share a common memory pool, reduce data traffic, and increase the speed of cooperation among the system components. At the lowest level, it is critical for CPU-and accelerator cores to have their own memory hierarchies (L1, L2, LLC). Shared memory pools could be very effective mechanism for coordinating a workflow across the heterogenous system components. Workflows can then evolve from a file-based sharing method to a shared memory model utilizing a high-speed fabric, e.g., CXL. Composability also addresses the sustainability issues of large computing systems by allowing for upgrades of individual parts in the system. The use of heterogenous components must expand from the current rack-level or board-level integration down to chiplet-based modules, and even System-on-Chip (SoC), depending on the scale and demands of a workflow. A standards-based coherent interconnect fabric is a key element that will allow innovations from different heterogeneous components to be mixed beyond the limitations of any single vendor and is also a key ingredient for an industry growth play. For board-level connections outside of the SMP fabric, the evolving CXL standards are a good match for this role as they support traditional I/O connect plus a scalable memory extension. CXL over the emerging UCIe connection standard offers the possibility to extend this value proposition to a chiplet-based ecosystem, where tight integration into the SMP fabric is not required. For extended reach, CXL over an optics standard would provide for even larger scale composable systems. The composable elements in a compute fabric need a distributed control structure for initialization, resource management, and workflow control. Open standards such as OFMF will play a critical role in the overall system management. The emergence of confidential computing as a paradigm for reducing the trusted computing base (TCB) of a computation is also essential for HPC and cloud. Open standards are required to enable confidential computing's trusted execution environments (TEEs) across heterogeneous elements. Security features will be required to address supply chain attacks, secure and trusted boot, authentication and attestation of each component, enable secure and confidential communication between the various components in the heterogenous system.

The slow-down of technology scaling combined with the exponential growth of modern machine learning and artificial intelligence models has created a demand for specialized accelerators, such as GPUs, ASICs, and field-programmable gate arrays (FPGAs). FPGAs can be reconfigured and have the potential to outperform other accelerators, while also being more energy-efficient, but are cumbersome to use with today's fractured landscape of tool flows. We propose the concept of an operation set architecture to overcome the current incompatibilities and hurdles in using DNN-to-FPGA compilers by combining existing specialized frameworks into one organic compiler that also allows the efficient and automatic re-use of existing community tools. Furthermore, we demonstrate that mixing different existing frameworks can increase the efficiency by more than an order of magnitude.

Field Programmable Gate Arrays (FPGAs) are increasingly being used in data centers and the cloud due to their potential to accelerate certain workloads as well as for their architectural flexibility, since they can be used as accelerators, smart-NICs, or stand-alone processors. To meet the challenges posed by these new use cases, FPGAs are quickly evolving in terms of their capabilities and organization. The utilization of High Bandwidth Memory (HBM) in FPGA devices is one recent example of such a trend. In this article, we study the potential of FPGAs equipped with HBM from a data analytics perspective. We consider three workloads common in analytics-oriented databases and implement them on an FPGA showing in which cases they benefit from HBM: range selection, hash join, and stochastic gradient descent for linear model training. We integrate our designs into a columnar database (MonetDB) and show the trade-offs arising from the integration related to data movement and partitioning. We consider two possible configurations of the HBM, using a single and a dual clock version design. With the right design, FPGA+HBM-based solutions are able to surpass the highest performance provided by either a two-socket POWER9 1 system or a 14-core Xeon 2 E5 by up to 5.9× (range selection), 18.3× (hash join), and 6.1× (SGD).

Ongoing climate change calls for fast and accurate weather and climate modeling. However, when solving large-scale weather prediction simulations, state-of-the-art CPU and GPU implementations suffer from limited performance and high energy consumption. These implementations are dominated by complex irregular memory access patterns and low arithmetic intensity that pose fundamental challenges to acceleration. To overcome these challenges, we propose and evaluate the use of near-memory acceleration using a reconfigurable fabric with high-bandwidth memory (HBM). We focus on compound stencils that are fundamental kernels in weather prediction models. By using high-level synthesis techniques, we develop NERO, an field-programmable gate array+HBM-based accelerator connected through Open Coherent Accelerator Processor Interface to an IBM POWER9 host system. Our experimental results show that NERO outperforms a 16-core POWER9 system by \( 5.3\times \) and \( 12.7\times \) when running two different compound stencil kernels. NERO reduces the energy consumption by \( 12\times \) and \( 35\times \) for the same two kernels over the POWER9 system with an energy efficiency of 1.61 GFLOPS/W and 21.01 GFLOPS/W. We conclude that employing near-memory acceleration solutions for weather prediction modeling is promising as a means to achieve both high performance and high energy efficiency.

High-Performance Big Data Analytics (HPDA) applications are characterized by huge volumes of distributed and heterogeneous data that require efficient computation for knowledge extraction and decision making. Designers are moving towards a tight integration of computing systems combining HPC, Cloud, and IoT solutions with artificial intelligence (AI). Matching the application and data requirements with the characteristics of the underlying hardware is a key element to improve the predictions thanks to high performance and better use of resources. We present EVEREST, a novel H2020 project started on October 1, 2020, that aims at developing a holistic environment for the co-design of HPDA applications on heterogeneous, distributed, and secure platforms. EVEREST focuses on programmability issues through a data-driven design approach, the use of hardware-accelerated AI, and an efficient runtime monitoring with virtualization support. In the different stages, EVEREST combines state-of-the-art programming models, emerging communication standards, and novel domain-specific extensions. We describe the EVEREST approach and the use cases that drive our research.

We present an architecture for field-programmable gate arrays (FPGAs) to expose RESTful web services. This architecture allows clients to access accelerated web services from any platform and programming language that can perform RESTful API calls. By using this architecture, the client's application benefits from a high throughput and low latency web service interface. Traditionally, FPGAs are deployed in CPU-centric infrastructures as worker devices in the form of accelerators. However, for FPGA-centric applications, the overhead of a host CPU diminishes the performance, scalability and energy efficiency. cloudFPGA solves these issues by deploying FPGAs as standalone, disaggregated resources in the DC. Building on top of the cloudFPGA platform, the presented architecture simplifies the integration of FPGA-accelerated functions with cloud applications. A configurable hardware block that can be generated from an OpenAPI-based specification of the web service is used to deploy an FPGA-based application. We compare a natural language processing (NLP) application that is exposed as a web service using the traditional server infrastructure and our RESTful service layer. Measurements show an improvement of 20x in terms of throughput and 4x reduction in mean latency.

The evolution of cloud applications into loosely-coupled microservices opens new opportunities for hardware accelerators to improve workload performance. Existing accelerator techniques for cloud sacrifice the consolidation benefits of microservices. This paper presents CloudiFi, a framework to deploy and compare accelerators as a cloud service. We evaluate our framework in the context of a financial workload and present early results indicating up to 485 x gains in microservice response time.

With Moore's law progressively running out of steam, heterogeneous computing architectures have been powering the top supercomputers in the world for many years and are now finding broader adoption across the industry. The trend towards sustainable computing also requires domain-specific heterogeneous hardware architectures, which promise further gains in energy efficiency. At the same time, today's high performance computing applications have evolved from monolithic simulations in a single domain to multidisciplinary complex workflows. In this paper, we explore how these trends affect system design decisions and what this means for future computing system architectures.

The slowdown of Moore's law and the end of Dennard scaling created a demand for specialized accelerators, including Field Programmable Gate Arrays (FPGAs), in cloud data centers. At the same time, compute resources are increasingly consumed via public and private clouds and traditional applications are modernized using scalable microservices and Function-as-a-Service (FaaS) offerings. Nonetheless, true FaaS based on FPGAs or other accelerators is virtually absent from the offering catalogs of all major cloud providers. In addition, FPGA applications are typically coded in a monolithic fashion, due to device and vendor specific dependencies, which reduces the portability and usability of FPGA cloud offerings further. However, FPGA-based FaaS can improve execution efficiency and minimize (tail-) latencies while decreasing costs. We propose a novel system architecture, called Mantle, that uses disaggregated FPGAs to enable scalable, usable, portable and efficient FaaS offerings for FPGAs. Our experimental results demonstrate a significant reduction of end-to-end service provisioning time to below 7 seconds and an increase in execution efficiency by a factor of 4 with negligible overhead.

System security is currently a main focus area for all IT infrastructure providers. New system features like pervasive encryption, the transition to cloud-based offerings, and the demand for quantum-safe platforms demand increased cryptographic performance as well as more cryptographic agility. The new IBM 4769 Cryptographic Coprocessor addresses these trends. It brings performance improvements that match the requirements of the new IBM z15. A combination of newly available features allows IBM z15 to scale to greater than 5,000 Virtual Hardware secure modules per system and makes it suitable to support virtualized client environments such as cloud-scale datacenters. To meet the dense packaging and energy requirements of those data centers, the form factor and power consumption of the card were reduced significantly. The card also offers an expanded set of algorithms to support state-of-the-art as well as future workloads. For the first time, the user interface provides access to a selected set of quantum-safe algorithms. Infrastructure extensions add hardware-embedded, attestation-friendly trusted boot services, which improve system resiliency by providing hardware enabled measurements of the secure and trusted boot process. These extensions simultaneously simplify the security certifications built on them. This article provides an overview of the IBM 4769 cryptographic coprocessor, highlighting security characteristics, internal hardware, form factor, and enhanced firmware.

The performance improvement rate of conventional von Neumann processors has slowed as Moore's Law grinds to an economic halt, giving rise to a new age of heterogeneity for energy-efficient computing. Extending processors with finely tunable precision instructions have emerged as a form of heterogeneity that tradeoffs computation precision with power consumption. However, the prolonged design time due to customization of the supported framework for a system-on-a-chip may counteract the advantages of transprecision computing. We propose XwattPilot, a system aiming at accelerating the transprecision software development of low-power processors using cloud technology. We show that the total energy-to-solution can be significantly decreased by using transprecision computations, whereas the proposed system can accelerate the energy-efficiency evaluation runtime by 10.3×.

Advances in Next Generation Sequencing (NGS) technologies have caused the proliferation of genomic applications to detect DNA mutations and guide personalized medicine. These applications have an enormous computational cost due to the large amount of genomic data they process. Although leveraging FPGAs can improve the processing time of such amount of data, the limited memory capacity of FPGAs often restricts the potential gains. To overcome this limitation, IBM CAPI (Coherent Accelerator Processor Interface) supported platforms provide FPGAs with direct access to the CPU memory. This paper proposes a hardware/software co-design for k-mer counting, one of the most time-consuming phases of genomic applications. The proposed co-design targets CAPI-enabled FPGAs and is integrated into SMUFIN, a state-of-the-art reference-free method for finding DNA mutations. Results show that the proposed co-design outperforms the CPU-only design by a factor of 2.14×, it consumes 2.93× less energy, and it requires 1.57× less memory.

Specialized accelerators for tensor-operations, such as blocked-matrix operations and multi-dimensional convolutions, have been emerged as powerful architecture choices for high-performance Deep-Learning computing. The rapid development of frameworks, models, and precision options challenges the adaptability of such tensor-accelerators since the adaptation to new requirements incurs significant engineering costs. Programmable tensor accelerators offer a promising alternative by allowing reconfiguration of a virtual architecture that overlays on top of the physical FPGA configurable fabric. We propose an overlay ({\tau}-VTA) and an optimization method guided by agile-inspired auto-tuning techniques. We achieve higher performance and faster convergence than state-of-art.

Ongoing climate change calls for fast and accurate weather and climate modeling. However, when solving large-scale weather prediction simulations, state-of-the-art CPU and GPU implementations suffer from limited performance and high energy consumption. These implementations are dominated by complex irregular memory access patterns and low arithmetic intensity that pose fundamental challenges to acceleration. To overcome these challenges, we propose and evaluate the use of near-memory acceleration using a reconfigurable fabric with high-bandwidth memory (HBM). We focus on compound stencils that are fundamental kernels in weather prediction models. By using high-level synthesis techniques, we develop NERO, an FPGA+HBM-based accelerator connected through IBM CAPI2 (Coherent Accelerator Processor Interface) to an IBM POWER9 host system. Our experimental results show that NERO outperforms a 16-core POWER9 system by 4.2x and 8.3x when running two different compound stencil kernels. NERO reduces the energy consumption by 22x and 29x for the same two kernels over the POWER9 system with an energy efficiency of 1.5 GFLOPS/Watt and 17.3 GFLOPS/Watt. We conclude that employing near-memory acceleration solutions for weather prediction modeling is promising as a means to achieve both high performance and high energy efficiency.

Messaging frameworks are prevalent in distributed software systems as they simplify the exchange of data between applications running on different platforms. Messages are serialized and deserialized to transform user data structures into binary data for sending across the network. In a similar spirit, modern heterogenous computing platorms typically containing FPGAs can also profit from such messaging frameworks. Efficient compact serialization of data types is needed to take full advantage of the high throughput communication links. Transprecision extensions of FPGA replace native data types with novel low-precision data types. Transprecision extensions are shown to result in less-compact serializations. In this paper, we present a messaging framework for transprecision hardware accelerators which supports messages expressed in Protobuf and automatically generates serialization and deserialization functions. Our messageing framework leverages the varint encoding of Protocol buffers for efficient serialization and deserialization of transprecision data types. Evaluation results show that it can increase the effective throughput of OpenCAPI by up to 7.4× for various transprecision data-types.

FPGA-based data processing in datacenters is increasing in popularity due to the demands of modern workloads and the resulting need for specialization in hardware. Driven by this trend, vendors are rapidly adapting reconfigurable devices to suit data and compute intensive workloads. Inclusion of High Bandwidth Memory (HBM) in FPGA devices is a recent example. HBM promises overcoming the bandwidth bottleneck, often faced by FPGA-based accelerators due to their throughput oriented design. In this paper, we study the usage and benefits of HBM on FPGAs from a data analytics perspective. We consider three workloads that are often performed in analytics oriented databases and implement them on FPGA showing in which cases they benefit from HBM: range selection, hash join, and stochastic gradient descent for linear model training. We integrate our designs into a columnar database (MonetDB) and show the trade-offs arising from the integration related to data movement and partitioning. In certain cases, FPGA+HBM based solutions are able to surpass the highest performance provided by either a 2-socket POWER9 system or a 14-core XeonE5 by up to 1.8x (selection), 12.9x (join), and 3.2x (SGD).