Program Synthesis from Natural Language Using Recurrent Neural Networks

O entimes, a programmer may have di culty implementing a desired operation. Even when the programmer can describe the goal in English, it can be di cult to translate into code. Existing resources, such as question-and-answer websites, tabulate speci c operations that someone has wanted to perform in the past, but they are not e ective in generalizing to new tasks, to compound tasks that require combining previous questions, or sometimes even to variations of listed tasks.Our goal is to make programming easier and more productive by le ing programmers use their own words and concepts to express the intended operation, rather than forcing them to accommodate the machine by memorizing its grammar. We have built a system that lets a programmer describe a desired operation in natural language, then automatically translates it to a programming language for review and approval by the programmer. Our system, Tellina, does the translation using recurrent neural networks (RNNs), a state-of-the-art natural language processing technique that we augmented with slot (argument) lling and other enhancements. We evaluated Tellina in the context of shell scripting. We trained Tellina’s RNNs on textual descriptions of le system operations and bash one-liners, scraped from the web. Although recovering completely correct commands is challenging, Tellina achieves top-3 accuracy of 80% for producing the correct command structure. In a controlled study, programmers who had access to Tellina outperformed those who did not, even when Tellina’s predictions were not completely correct, to a statistically signi cant degree.