From High-Level Deep Network Models to FPGA Acceleration

Deep Neural Networks (DNNs) are compute-intensive learning models with growing applicability in a wide range of domains. FPGAs are a compelling alternative to accelerate deep networks. However, using FPGAs for DNN acceleration is challenging since it requires long design cycles and expertise in hardware design. This work takes on this challenge, but instead of designing just an accelerator for a particular DNN model, it devises DNNWEAVER; a framework that automatically generates a synthesizable accelerator for a given (DNN, FPGA) pair using hand-optimized templates. DNNWEAVER uses Caffe [1] as the programming interface to provide a high level of abstraction to the programmers. We use DNNWEAVER to generate accelerators for the combinations of five different deep networks and two different FPGAs, Xilinx Zynq and Altera Stratix V. We rigorously compare the generated accelerators to multicore CPUs (ARM Cortex A15 and Xeon E3) and many-core GPUs (Tegra K1, GTX 650 Ti, and Tesla K40). In comparison, the generated accelerators deliver superior performance compared to CPUs and superior efficiency compared to GPUs; without requiring the programmers to participate in the arduous task of hardware design.