Test-Driven Development for Code Generation
CoRR(2024)
摘要
Large language models (LLMs) like GPT4, have shown proficiency in generating
code snippets from problem statements. Traditionally software development by
humans followed a similar methodology of writing code from problem statements
or requirements. However, in the past, there have been several studies that
have shown the value of test-driven development (TDD) where humans write tests
based on problem statements before the code for the functionality is written.
In the context of LLM-based code generation, one obvious benefit of TDD is that
the developer then knows for sure if the generated code has passed all the
given tests or not. Therefore, in this paper, we want to empirically evaluate
the hypothesis: giving the problem statements and tests as input to GPT4 is
better than just giving the problem statement as input. To test our hypothesis,
we build a framework TGen. In our experiments on the MBPP, HumanEval and
CodeChef datasets, we consistently find that including tests solves more
programming problems than not including them. Thus we show that TDD is a better
development model than just using a problem statement when using GPT4 for code
generation tasks.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要