Analyzing CodeBERT’s Performance on Natural Language Code Search

Daya Guo,Shuo Ren,Shuai Lu,Zhangyin Feng, Duyu, Shujie Tang, Long Liu, Nan Zhou, Alexey Duan, Svy-64 Shengyu, Michele Fu, Shao Tufano, Kun, Colin B Deng, Dawn Clement, Neel Drain, Sundare-65,Emily Hill,Lori Pollock, Hamel Husain, Hongduo Wu, Tiferet Gazit, Miltiadis, Allamanis Marc, Brockschmidt., Code-67,Yinhan Liu,Myle Ott,Naman Goyal,Jingfei Du, Mandar Joshi,Danqi Chen,Omer Levy,Mike Lewis, Alexey Junjie Huang,A. Svyatkovskiy, Colin B Blanco, Clément, Dong Zhou,Linjun Shou,Long Zhou, Michele Tu-69 fano,Ming Gong,Ming Zhou,Nan Duan,Shao Kun Deng, Sheng-Qiao Fu,Thomas Wolf,Lysandre Debut, Julien Victor Sanh, Clement Chaumond, Anthony Delangue, Pier-33 Moi, Tim ric Cistac, Rémi Rault, Morgan Louf, Funtow-90 Joe, Sam Davison, Patrick Shleifer, V. Platen, Clara Ma,Yacine Jernite, J. Plu,Canwen Xu, Teven Le Scao,Sylvain Gugger,Mariama Drame,Yonghui Wu,Mike Schuster,Zhifeng Chen,Quoc V. Le,Mohammad Norouzi,Wolfgang Macherey, Maxim, Yuan Krikun, Qin Cao, Klaus Gao, J. Macherey, Apurva Klingner, M. Shah, Xiaobing Johnson, ukasz Liu, Stephan Kaiser, Yoshikiyo Gouws, Kato,Ziyu Yao, Jayavardhan Reddy Peddamail,Pengcheng Yin,Bowen Deng,Edgar Chen,Jian Zhang, Xu Wang,Hongyu Zhang,Hailong Sun

semanticscholar（2022）

被引用0|浏览19

摘要

Large language models such as CodeBERT 001 (Feng et al., 2020) perform very well on tasks 002 such as natural language code search. We show 003 that this is most likely due to the high token 004 overlap and similarity between the queries and 005 the code in datasets obtained from large code-006 bases, rather than any deeper understanding of 007 the syntax or semantics of the query or code. 008

更多

查看译文

上传PDF