Evaluating explainable artificial intelligence methods for multi-label deep learning classification tasks in remote sensing

•We evaluated quantitatively and qualitatively different aspects of ten XAI methods.•We employed various metrics in order to assess XAI methods’ performance.•XAI methods for multi-label classification tasks in BigEarthNet and SEN12MS datasets.•Extracted significant insights regarding models’ decisions and datasets’ composition.•Occlusion, LIME and Grad-CAM were the most interpretable methods.