Compiling the uncompilable: A case for shell script compilation

Abstract Shells, as command interpreters, are the classical way for humans to interact with computing systems, and modern shell features have extended this basic function- ality with higher-level programming,language constructs. Although implementing compilation in these shell languages is generally unprofitable and intractable, many advantages, such as isolation, filesystem abstraction, security, portability, paral- lelization and locality optimization are possible, using standard compilation tech- niques. While compilation is not possible for all scripts, there exist shell scripts of a class that are, in practice, both profitable and tractable to compile and execute. This class of scripts is prevalent in the scientific computing community, where scripts are commonly,used to automate data processing sequences. We describe a prototype shell compilation and implementation for these scripts, noting advantages and chal- lenges, and illustrating the significant performance potential available. Our results show that shell compilation is a viable means of automatically identifying and ex- ploiting high-level program parallelism using existing sequential script specification and without requiring reimplementation. Key words: programming languages, shell languages, domain-specific language,