Open-Source Reinforcement Learning Environments Implemented in MuJoCo with Franka Manipulator

This paper presents three open-source reinforcement learning environmentsdeveloped on the MuJoCo physics engine with the Franka Emika Panda arm inMuJoCo Menagerie. Three representative tasks, push, slide, and pick-and-place,are implemented through the Gymnasium Robotics API, which inherits from thecore of Gymnasium. Both the sparse binary and dense rewards are supported, andthe observation space contains the keys of desired and achieved goals to followthe Multi-Goal Reinforcement Learning framework. Three different off-policyalgorithms are used to validate the simulation attributes to ensure thefidelity of all tasks, and benchmark results are also given. Each environmentand task are defined in a clean way, and the main parameters for modifying theenvironment are preserved to reflect the main difference. The repository,including all environments, is available athttps://github.com/zichunxx/panda_mujoco_gym.