Measuring Sample Efficiency and Generalization in Reinforcement Learning Benchmarks: NeurIPS 2020 Procgen Benchmark

Sharada Mohantya b,c mohanty@aicrowd.com Jyotish Poonganama b jyotish@aicrowd.com Adrien Gaidond e adrien.gaidon@tri.global Andrey Kolobovd f akolobov@microsoft.com Blake Wulfed e blake.wulfe@tri.global Dipam Chakrabortyd c dipam@aicrowd.com Graz̆vydas S̆emetulskisd g grazvydas@threethirds.ai João Schapked h joaoschapke@gmail.com Jonas Kubiliusd g jonas@threethirds.ai Jurgis Paükonisd g jurgis@threethirds.ai Linas Klimasd g linas@threethirds.ai Matthew Hausknechtd f matthew.hausknecht@microsoft.com Patrick MacAlpined f patmac@gmail.com Quang Nhat Trand i quangtran@temple.edu Thomas Tumield j ttumiel@gmail.com Xiaocheng Tangd k xiaochengtang@didiglobal.com Xinwei Chend j o.xlnwel@outlook.com Christopher Hessel csh@openai.com Jacob Hiltonl jhilton@openai.com William Hebgen Gussl wguss@openai.com Sahika Genc m sahika@amazon.com John Schulmanl joschu@openai.com Karl Cobbe l karl@openai.com