Dynamic out of Core Join Processing in Symmetric Multiprocessors

The use of clusters of Symmetric Multiprocessor (SMP) configurations in database processing has become a key factor in allowing greater scalability. It has also posed many challenges in the implementation of one of the most costly operations within relational algebra: the join operation. When massive data is involved, usually the join cannot be performed in-memory and is processed out of core. In this case, performance depends on an effective use of the memory hierarchy, such that I/O and memory contention are minimized. In this paper we propose a parallel algorithm for out of core join processing that dynamically adapts its behavior to the resources available in the system. We evaluate and compare our proposal against other parallel approaches in a real SMP cluster in a major commercial database, the IBMR® DB2R® Universal Database™ product (DB2 UDB). Results show that our proposal outperforms previous work significantly.