Automatic Decomposition of a Sequential Algorithm for MapReduce Frameworks

Effective programming of parallel architectures has always been a difficult t ask. T o d ate, programming languages and technologies have been developed that simplify the programmer’s work, but do not make parallelization automatic. MapReduce is a model of programming for the development of large-scale computations with intensive use of data. There are many frameworks where the implementation of this paradigm has been recently developed. There is a need to rewrite existing serial code to use the frameworks listed. The researcher must be familiar with the problems of parallelization, the API of the framework, and also have considerable experience. This prompted us to develop a new tool that automatically translates sequential programs into ready-made versions suitable for execution in the MapReduce paradigm. The code fragment from the serial version is converted in two stages. At the first stage, the synthesis of the program, the functional specification, was made. It was necessary to find information about the calculation structure for each block of code. The result was stored as a high-level intermediate language, reminiscent of the program format for MapReduce frameworks. Checking for semantic equivalence to the original has done using the proof of the theorem. At the second stage, executable code is created, which was the result of generation from a sequential program using the Hadoop or Spark instruction set. Creating a parallelizing compiler is one way to solve this problem. This will allow you to translate code written in a different paradigm (for example, imperative code) into a parallel version for the framework. Classical compilers, such as logical plan-to-physical compilers, use pattern matching rules. The compiler contains a set of rules that identify different patterns of code input (for example, list-sequential looping) and transform consistent code.