Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Mode

Heuristics are indispensable for tackling complex search and optimization problems. However, manual heuristic design is tedious and demands significant human intuition and experience. This paper introduces Evolution of Heuristic (EoH), a novel paradigm that leverages the synergy between Large Language Models (LLMs) and Evolutionary Computation (EC) for Automatic Heuristic Design (AHD). EoH represents heuristic ideas through linguistic descriptions, termed thoughts, generated by LLMs, which are then translated into executable code representations. The coevolution of thoughts and codes within an evolutionary framework offers superior AHD performance while mitigating computational expenses. Comprehensive evaluations on three types of combinatorial optimization benchmarks demonstrate EoH's outperformance against existing AHD methods. Notably, EoH surpasses FunSearch, a concurrent work focus on code evolution, identifying superior heuristics with significantly fewer computational budgets (i.e., queries to LLMs) on online bin packing problem. To foster reproducibility and accessibility, the source code is https://github.com/FeiLiu36/EoH.