Scaling Ordered Stream Processing on Shared-Memory Multicores

Many modern applications require realtime processing of large volumes of high speed data. Such data processing can be specified in the form of a dataflow graph that exposes multiple opportunities for parallelizing its execution, in the form of data, pipeline and task parallelism. This paper focuses on the problem of effectively parallelizing ordered streaming computations by exploiting low overhead, shared memory parallelism on a multicore machine. We propose an adaptive runtime that dynamically maps the exposed parallelism in a streaming computation to that of the machine using scheduling heuristics. Further, we address some key problems in effectively realizing ordered data parallelism. We propose a new approach for parallelizing partitioned stateful operators that can handle load imbalance across partitions effectively and mostly avoid delays due to ordering constraints. We also present a low latency, non-blocking concurrent data structure to order outputs produced by concurrent workers on an operator. Finally, we perform an in-depth empirical evaluation illustrating the trade-offs and effectiveness of our concurrent data-structures and scheduling heuristics using micro-benchmarks and on the TPCx-BB benchmark.