VCSN: Virtual Circuit-Switching Network for Flexible and Simple-to-Operate Communication in HPC FPGA Cluster

FPGA clusters promise to play a critical role in high-performance computing (HPC) systems in the near future due to their flexibility and high power efficiency. The operation of large-scale general-purpose FPGA clusters on which multiple users run diverse applications requires a mechanism that provides an easily divided and reconfigured network topology. This paper proposes a Virtual Circuit-Switching Network (VCSN) that provides an arbitrary reconfigurable network topology and simple-to-operate network system among FPGA nodes. Users can operate communication as if a circuit-switching network through virtualization. This paper demonstrates that VCSN with 100 Gbps Ethernet achieves highly-efficient point-to-point communication among FPGAs due to its unique and efficient communication protocol. This paper compares VCSN with a direct connection network (DCN) that connects FPGAs directly. We also show a concrete procedure to realize collective communication on the FPGA cluster with VCSN. We demonstrate that the redundant virtual topology provided by VCSN can accelerate collective communication with simple operations on our FPGA cluster. Furthermore, based on experimental results, we model communication by DCN and VCSN in a large FPGA cluster and estimate the communication performance. The result shows that VCSN can perform gather communication up to about 1.97 times faster than DCN.