Topics in Parallel and Distributed Computing

The computing world is changing and all devices—from mobile phones and personal computers to high-performance supercomputers—are becoming parallel. At the same time, the efficient usage of all the opportunities offered by modern computing systems represents a global challenge. Using full potential of parallel computing systems and distributed computing resources requires new knowledge, skills and abilities, where one of the main roles belongs to understanding key properties of parallel algorithms. What are these properties? What should be discovered and expressed explicitly in existing algorithms when a new parallel architecture appears? How to ensure efficient implementation of an algorithm on a particular parallel computing platform? All these as well as many other issues are addressed in this chapter. The idea that we use in our educational practice is to split a description of an algorithm into two parts. The first part describes algorithms and their properties. The second part is dedicated to describing particular aspects of their implementation on various computing platforms. This division is made intentionally to highlight the machine-independent properties of algorithms and to describe them separately from a number of issues related to the subsequent stages of programming and executing the resulting programs. Relevant core courses: Data Structures and Algorithms, Second Programming Course in the Introductory Sequence. Relevant PDC topics: Parallel algorithms, computer architectures, parallel programming paradigms and notations, performance, efficiency, scalability, locality. Learning outcomes: Faculty staff mastering the material in this chapter should be able to: • Understand basic concepts of parallelism in algorithms and programs. • Detect parallel (information) structure of algorithms. V. Voevodin ( ) · A. Antonov · V. Voevodin Lomonosov Moscow State University, Moscow, Russia e-mail: voevodin@parallel.ru; asa@parallel.ru; vadim@parallel.ru © Springer International Publishing AG, part of Springer Nature 2018 S. K. Prasad et al. (eds.), Topics in Parallel and Distributed Computing, https://doi.org/10.1007/978-3-319-93109-8_2 23 24 V. Voevodin et al. • Understand deep relationship between properties of algorithms and features of computer architectures. • Identify main features and properties of algorithms and programs affecting performance and scalability of applications. • Use proper algorithms for different types of computer architectures. Context for use: This chapter has to touch all the main areas of computer science and engineering: Architecture, Programming, Algorithms and Crosscutting topics. The primary area is Algorithms but these materials should be taught after learning the fundamentals of computer architecture and programming technologies. Materials of the chapter can be easily adapted for use in core, advanced or elective courses within bachelor’s or master’s curricula.