Decision Transformer (DT) is a recently proposed architecture for Reinforcement Learning that frames the decision-making process as an auto-regressive sequence modeling problem and uses a Transformer model to predict the next action in a sequence of states, actions, and rewards. In this paper, we analyze how crucial the Transformer model is in the complete DT architecture on continuous control tasks. Namely, we replace the Transformer by an LSTM model while keeping the other parts unchanged to obtain what we call a Decision LSTM model. We compare it to DT on continuous control tasks, including pendulum swing-up and stabilization, in simulation and on physical hardware. Our experiments show that DT struggles with continuous control problems, such as inverted pendulum and Furuta pendulum stabilization. On the other hand, the proposed Decision LSTM is able to achieve expert-level performance on these tasks, in addition to learning a swing-up controller on the real system. These results suggest that the strength of the Decision Transformer for continuous control tasks may lie in the overall sequential modeling architecture and not in the Transformer per se.

While many current object tracking approaches focus on tracking humans or traffic situations, and not include robotic models, in this work we specifically focus on the task of object tracking in a robotic assembly environment. For this, we outline the specific challenges and chances of object tracking for the assembly purpose and then create a dataset in a PyBullet simulation environment, where a robot arm picks, moves and places a block in the scene. We then evaluate the tracking performance of the AR tracking library VisionLib and conclude that it is not suitable for tracking objects in robotic assembly. Finally, we outline further research topics which are important for creating a good object tracker in robotic assembly.