3Dmaviz: a Visual Analytics Framework for Exploring DRL-based 3D Multi-Agent Environments

Deep reinforcement learning (DRL) has achieved remarkable success in robotic autonomous control applications, owing to its superior performance. While existing visual analytics methods primarily focus on 2D single-agent environments, research on 3D multi-agent environments remains limited. In these environments, the increased dimensionality of agent action spaces and the complexity of dynamic interactions result in higher-dimensional data, demanding more sophisticated visualization techniques. This paper presents 3DMAViz, a visual analytics system designed to offer comprehensive insights into the training data of DRL models in 3D multi-agent environments. The system aims to enhance the understanding, diagnosis, and evaluation of these models, thereby improving their interpretability. To achieve this, 3DMAViz introduces an innovative ring target view that enables effective analysis of agent strategies through circular and multi-level visualizations, particularly in environments with increased dimensionality of agent action spaces. Additionally, it integrates a clustering and dimensionality reduction method with a deep Gaussian process model to classify high-dimensional patterns and quantify data correlations, thus improving the efficiency of identifying high-value information. Through case studies conducted in a 3D multi-agent simulation environment with domain researchers, we validate the effectiveness of 3DMAViz.