MapCheckReduce: an Improved MapReduce Computing Model for Imprecise Applications

Optimizing the Map Phase is an important way to reduce the MapReduce (MR) job runtime. The common way for such studies is to design more efficient scheduling policies. However, there exists a kind of MapReduce applications, named imprecise applications, where the reduce phase can be completed based on the outputs of part of the map tasks. According to the feature of imprecise applications, we propose an improved MapReduce model, named MapCheckReduce (MCR). MCR can terminate the map process when the requirements of imprecise applications are satisfied. Compared to MR, a Check mechanism and a set of extended programming interfaces are added to MCR. The Check mechanism can receive and analyze messages submitted by mappers and then determine whether to terminate the map phase. The programming interfaces can be used by the programmers of imprecise applications to define the termination conditions of the map phase. The MCR prototype has been implemented and experiment results verify the feasibility and effectiveness of MCR.