Complete Provenance for Application Experiments with Containers and Hardware Interface Metadata

Repeatability and reproducibility are important for science, but HPC application experiments have complex and changing software stacks and runtime environments which make this difficult to attain. This paper describes a system for combining containers and system hardware environment capture to fully capture an application experiment's provenance; the clear interfaces to hardware exposed by containers are a key element of this approach. The evaluation of this system demonstrates that this combination enables effective analysis of application dependencies, increasing the reproducibility of experimental results in HPC systems. It also demonstrates that this technique can provide a basis for Scientific Development Operations that result in verifiable application experiments.