Stream Processing Performance for Blue Gene/P Supercomputer

Stream processing systems are designed to support applications that use real time data. Examples of streaming applications include security agencies processing data from communications media, battlefield management systems for military operations, consumer fraud detection based on online transactions, and automated trading based on financial market data. Many stream processing applications are faced with the challenge of increasingly large volumes of data and the requirement to deliver low-latency responses predicated by analysis of that data. In this paper, we assess the applicability of the Blue Gene architecture for stream computing applications. This work is part of a larger effort to demonstrate the efficacy of using a Blue Gene for streaming applications. Blue Gene supercomputers provide a high-bandwidth low-latency network connecting a set of I/O and compute nodes. We examine Blue Genepsilas suitability for stream computing applications by assessing its messaging capability for typical stream computing messaging workloads. In particular, this paper presents results from micro-benchmarks we used to evaluate the raw performance of Blue Gene/P (Blue Gene/P) supercomputer under loads produced by high volumes of streaming data. We measure the performance of data streams that originate outside the supercomputer, are directed through the I/O nodes to the compute nodes and then terminate outside. Our performance experiments demonstrate that the Blue Gene/P hardware delivers low-latency and high-throughput capability in a manner usable by streaming applications.