A Generator for Creating Streaming Continuous Optimisation Benchmarks

In many real-world scenarios, instances arrive in a stream whose characteristics change over time, i.e. the stream is subject to data-drift. The ability to detect and respond to drift is crucial to optimise performance over the stream. However, there is currently no suitable stream-generator on which to evaluate the aforementioned methods. We address this by proposing SCOBench: a fully configurable generator that can generate streams of instances with different drift patterns specified by the user. In contrast to existing stream generators that define drift as a shift in the location of the optima over time of a single instance, SCOBench generates a stream of new instances from multiple functions, while drift between consecutive instances can be defined according to a change in three different metrics (landscape features, probing-trajectories and optima location). We verify that SCOBench correctly generates the intended drift patterns, then demonstrate that algorithms perform differently over streams, highlighting the need for drift detection and algorithm-selection per instance. SCOBench is fully integrated with IOHexperimenter for ease of use. All code and documentation are available as open-source.