Model-based Reinforcement Learning for Robot-Based Laser Material Processing

Articulated robotic arms in laser material processing require precise motion planning. Traditional motion planning methods face challenges in trajectory accuracy. This study demonstrates model-based reinforcement learning as an effective approach for motion planning of these robotic arms. The process involves training a neural network trajectory model based on Pilz Industrial Motion Planner, followed by training an agent to optimize motion by adjusting joint velocities. The study compares Proximal Policy Optimization and Soft Actor-Critic algorithms to the baseline Pilz motion plan. Results show that model-based reinforcement learning improves accuracy in x-direction, reducing mean absolute error to 1.75 x 10(-3) m from 6.37 x 10(-3) m. However, it slightly increases z-direction mean absolute error, from 6 x 10(-6) m to 2.5 x 10(-4) m. This leads to an increase in on-surface beam radius, from 2.9 x 10(-5) m to 3.3 x 10(-5) m, and decrease in peak intensity of 22.77 % compared to baseline. These results highlight reinforcement learning's potential to enhance trajectory accuracy in motion planning, advancing robot-based laser material processing. (c) 2024 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0)