Code Generation with AlphaCodium: From Prompt Engineering to Flow Engineering
CoRR(2024)
摘要
Code generation problems differ from common natural language problems - they
require matching the exact syntax of the target language, identifying happy
paths and edge cases, paying attention to numerous small details in the problem
spec, and addressing other code-specific issues and requirements. Hence, many
of the optimizations and tricks that have been successful in natural language
generation may not be effective for code tasks. In this work, we propose a new
approach to code generation by LLMs, which we call AlphaCodium - a test-based,
multi-stage, code-oriented iterative flow, that improves the performances of
LLMs on code problems. We tested AlphaCodium on a challenging code generation
dataset called CodeContests, which includes competitive programming problems
from platforms such as Codeforces. The proposed flow consistently and
significantly improves results. On the validation set, for example, GPT-4
accuracy (pass@5) increased from 19
to 44
acquired in this work, we believe, are broadly applicable to general code
generation tasks. Full implementation is available at:
https://github.com/Codium-ai/AlphaCodium
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要