Maximum number of generations as a stopping criterion considered harmful

Evolutionary algorithms have been shown to be very effective in solving complex optimization problems. This has driven the research community in the development of novel, even more efficient evolutionary algorithms. The newly proposed algorithms need to be evaluated and compared with existing state-of-the-art algorithms, usually by employing benchmarks. However, comparing evolutionary algorithms is a complicated task, which involves many factors that must be considered to ensure a fair and unbiased comparison. In this paper, we focus on the impact of stopping criteria in the comparison process. Their job is to stop the algorithms in such a way that each algorithm has a fair opportunity to solve the problem. Although they are not given much attention, they play a vital role in the comparison process. In the paper, we compared different stopping criteria with different settings, to show their impact on the comparison results. The results show that stopping criteria play a vital role in the comparison, as they can produce statistically significant differences in the rankings of evolutionary algorithms. The experiments have shown that in one case an algorithm consumed 50 times more evaluations in a single generation, giving it a considerable advantage when max gen was used as the stopping criterion, which puts the validity of most published work in question.