Hierarchical automatic curriculum learning: Converting a sparse reward navigation task into dense reward.

•This paper proposed a Hierarchical Automatic Curriculum Learning (HACL) framework to address the sparse reward challenge.•The hierarchical and curriculum formulation convert the sparse reward tasks into dense reward, therefore alleviates the long-horizon difficulty.•HACL builds the hierarchy and curriculum stages automatically.•HACL achieves comparable or even better performances on three sparse reward tasks but with significant superiority in sample efficiency.•Extensive experiments show the advantages of three innovations in HACL.•Thanks very much for the reviewers comments and suggestion.