Systematic Construction, Execution, And Reproduction Of Complex Performance Benchmarks

In this work, we present the next generation of the Elba toolkit available under a Beta release, showing how we have used it for experimental research in computer systems using RUBBoS, a well-known n-tier system benchmark, as example. In particular, we show how we have leveraged milliScope - Elba toolkit's monitoring and instrumentation framework - to collect log data from benchmark executions at unprecedented fine-granularity, as well as how we have specified benchmark workflows with WED-Make - a declarative workflow language whose main characteristic is to facilitate the declaration of dependencies. We also show how to execute WED-Makefiles (i.e., workflow specifications written with WED-Make), and how we have successfully reproduced the experimental verification of the millibottleneck theory of performance bugs in multiple cloud environments and systems.