Transformers evolved to dominate the state of the art in almost any deep leaning domain, and usually involve very large models that are computationally expensive for training and inference. Smaller Transformer variants are still useful, particularly for a wide range of embedded applications. Existing compilers for FPGAs often lack support for core operations of the Transformer, such as the attention mechanism. In this work, we leverage the FINN framework to enable the automatic synthesis of custom-tailored FPGA accelerators for quantized Transformer models. We describe the hardware design of the scaled dot-product attention operation following the streaming dataflow paradigm and the integration into the compiler infrastructure of the FINN framework. We demonstrate a small-scale design space characterization exploring scaling behavior and resource utilization of our implementation, followed by two exemplary case studies, covering a radio signal classification use case as well as text generation with small GPTs, to demonstrate the end to end toolflow starting from quantization aware training up to deployment on the device. We evaluate our approach in terms of resource utilization, throughput, latency, accuracy and perplexity.
In this paper, we present a design methodology for multi-model accelerators considering the similar nodes between the two DNN computational graphs. First, we use a maximal common subgraph search algorithm to automatically extract the similarities between computational graphs. Then, we merge the common subgraphs and map the merged computation graph to hardware. We also present an LSTM-GRU hybrid accelerator without the need for redesigning and redeployment. Experimental results indicate that the hybrid accelerator achieves an average reduction of 42.3 % in the total logical resource consumption compared to the independent LSTM and GRU accelerators.
The routing architecture significantly impacts FPGA performance and area. Traditionally, routing resources are divided into connection blocks (CBs) and switch blocks (SBs) to implement interconnects based on high-fanin multiplexers (MUXes). The number of MUX levels is typically fixed in past studies, resulting in signals traversing a consistent number of MUXes within defined routing blocks. Although there is a lot of research focusing on the routing model, how MUXes implement these connections is less explored. In this paper, we propose a novel routing architecture based on multi-level MUXes, allowing for flexible driving relationships based on connections through various numbers of MUX levels. We define parameters to describe the multi-level MUX architecture, covering the number of MUX levels and the type and number of MUX fan-ins, and conduct experiments to investigate the effectiveness based on these parameters. In addition, we employ Bayesian Optimization (BO) for design space exploration based on the above parameters and wire distribution. The experimental result demonstrates that the optimized multi-level architecture achieves a 13.6 % reduction in critical path delay (CPD) with a 4.3 % overhead in total routing area compared to General Routing Block (GRB) architecture using VTR benchmarks. The multi-level MUX result also surpasses the BO searched result without the exploration of MUX arrangement, showing improvements of 6.9 % in CPD and 2.1 % in total routing area. Additionally, it shows 8.6 % and 5.0% improvement in CPD and routing area respectively than the versatile interconnection block (VIB) architecture.
Vendors design the Programmable Logic Fabric (PL) of an FPGA for robustness over a wide range of customer designs, use cases, and operating scenarios. These specifications cover for Process-Voltage-Temperature (PVT) variations occurring on the PL and are meant to ensure reliable timing margins on the FPGA. With increasing power densities, advanced packaging techniques and the use of PL for data-intensive computations, there have been efforts to understand the effects of voltage transients on the PL and their dependence on workloads. In this work, we propose a scalable methodology based on soft digital sensor (SDS) to characterize on-die voltage transients for compute-intensive workloads. This method is scalable with respect to test, measurement, and debugging compared to prior work. In addition, we show how the proposed methodology can be extended to study trends of timing margins and voltagefrequency headroom on FPGAs. This can improve the energy efficiency of designs running at a fixed operating point or using dynamic voltage-frequency scaling. Our results gathered from an AMD Versal VCK190 FPGA board running multiple highactivity workloads show incidents of large voltage droops of nearly 100 mV using the SDS. We also observe between 18-31% frequency headroom available across the higher activity designs analyzed in the presence of droop and jitter. These margins potentially allow designers to reliably push the operating voltage and frequency beyond the limits specified by the FPGA vendors, leading to much improved energy efficiencies.
Table-lookup inference, enabling convolutions via table lookups instead of multiply-accumulate (MAC) operations, shows promise for energy-efficient binarized convolutional neural networks (BCNNs). However, existing FPGA-based table-lookup BCNN designs consume considerable hardware and power due to the low binding level and LUT reuse. To address this, we propose EGSBConv, a novel convolution engine optimized for tablelookup inferences. EGSBConv leverages separable convolution and channel grouping to reduce fan-ins, facilitating kernel-level binding, while mitigating accuracy loss through internal channel extension. Each convolutional kernel is bound into a single table stored in BRAMs to reduce power consumption. Experiments on CIFAR-10 demonstrate 86% and 94% LUT reductions as well as 4.5x and 15.3x power efficiency improvements compared to state-of-the-art MAC-based and table-lookup-based BCNN accelerators, respectively.
