Interactive Parallel Workflows for Synchrotron Tomography

The constant stream of new users, samples, and experimental approaches at many light source beam lines means a single refined set of processing steps and parameters can seldom be re-used, and instead an interactive approach to exploring parameter space is required. But the many large data sets often generated during experiments mean that testing even one set of parameters is time intensive. We present two approaches to leverage parallel high performance computing to quickly and interactively explore parameter space for a large number of data sets, in the context of synchrotron tomography. The first approach uses a combination of template Jupyter notebooks with custom widgets, along with Papermill, and Dask. The second approach leverages the workflow infrastructure of Xi-Cam and Cam-Link, an ecosystem designed for real-time interactive exploration of data at user facilities.