Profiling Transport Performance For Big Data Transfer Over Dedicated Channels

The transfer of big data is increasingly supported by dedicated channels in high-performance networks. Transport protocols play a critical role in maximizing the link utilization of such high-speed connections. We propose a Transport Profile Generator (TPG) to characterize and enhance the end-to-end throughput performance of transport protocols. TPG automates the tuning of various transport-related parameters including socket options and protocol-specific configurations, and supports multiple data streams and multiple NIC-to-NIC connections. To instantiate the design of TPG, we use UDT as an example in the implementation and conduct extensive experiments of big data transfer over high-speed network channels to illustrate how existing transport protocols benefit from TPG in optimizing their performance.