Pink: A 1024-node Single-System Image Linux Cluster

This paper describes our experience of designing and building Pink, a 1024-node (2048 processor) Myrinet-based single-system image Linux cluster that was installed in January 2003 at the Los Alamos National Laboratory. At the time of its installation, Pink was the largest single-system image Linux cluster in the world, and was based entirely on open-source software - from the BIOS up. Pink was the proof-of-concept prototype for Lightning, a production 1408-node (2816 processor) cluster that will begin operation at LANL later this year. Lightning is currently number 6 on the Top500 list. In this paper we will examine the issues that were encountered and the problems that needed to be overcome in order to scale a cluster to this size. We will also present some performance numbers that demonstrate the scalability and manageability of the cluster software suite.