Open Source Project Running BSP on Ciel

In this report, I will review my implementation of Valiant’s Bulk Synchronous Parallel (BSP) model on top of the universal execution engine Ciel. The BSP model [1] is designed for large scale graph processing and uses parallelism to achieve high performance. It uses a vertex centric approach where a function is run on each vertex individually, and messages are passed between vertices. Ciel on the other hand is an execution engine capable of of coordinating a dynamic set of parallel tasks [2]. It therefore allows for the computation of iterative programs, which isn’t the case of most other execution engines such as MapReduce [3] or Dryad [4]. In the first section, I will review both the computational models of BSP, along with some of the modification introduced with the Pregel execution engine, and of Ciel. Then I will describe my implementation of BSP over Ciel and describe the optimizations I implemented. Finally I will evaluate my execution engine against Giraph [5], an open source BSP implementation running on top of Hadoop [6].