Measuring Optimiser Performance on a Conical Barrier Tree Benchmark

The common method for testing metaheuristic optimisation algorithms is to benchmark against problem test suites. However, existing benchmark problems limit the ability to analyse algorithm performance due to their inherent complexity. This paper proposes a novel benchmark, BTB, whose member functions have known geometric properties and critical point topologies. A given function in the benchmark is a realisation of a specified barrier tree in which funnel and basin geometries, and values and locations of all critical points are predetermined. We investigate the behaviour of two metaheuristics, PSO and DE, on the simplest manifestations of the framework, ONECONE and TWOCONES, and relate algorithm performance to a downhill walker reference algorithm. We study success rate, defined as the probability of optimal basin attainment, and inter-basin mobility. We find that local PSO is the slowest optimiser on the unimodal ONECONE but surpasses global PSO in all TWOCONES problems instances below 70 dimensions. DE is the best optimiser when basin difference depths are large but performance degrades as the differences become smaller. LPSO is the superior algorithm in the more difficult case where basins have similar depth. DE consistently finds the optimum basin when the basins have equal size and a large depth difference in all dimensions below 100D; the performance of LPSO falls away abruptly beyond 70D.