Classification methods based on deep neural networks (DNNs) lack interpretability, making them difficult to gain full trust in critical fields such as finance, healthcare, and law, which greatly limits their applications. Most existing studies focus on the interpretability of unimodal data, while challenges remain in the interpretability of multimodal data, especially when multimodal pattern recognition models are deployed on resource-constrained edge devices. To address this problem, this paper proposes a lightweight edge-oriented multimodal explainable image classification method based on visual attributes and decision-tree reasoning. The method integrates attributes extracted from different visual modalities, such as visible-light images and depth maps, into model training, and explains the decision process through visual attributes, hierarchical decision trees, and uncertainty-aware multimodal fusion. The attribute representation is compressed by global pooling, and the decision-tree reasoning module performs category-level inference with low computational overhead, making the framework compatible with lightweight backbones and energy-efficient edge inference. Although introducing interpretability usually leads to a decrease in model accuracy, the proposed method maintains good interpretability while achieving high classification accuracy. On three datasets, NYUDv2, SUN RGB-D, and RGB-NIR, the model achieves significantly improved accuracy compared with unimodal explainable methods and performance comparable to multimodal non-explainable models. Additional edge-oriented analysis further shows that replacing the backbone with lightweight CNNs can reduce theoretical parameter and FLOP costs, indicating the potential of the proposed framework for lightweight and energy-efficient edge pattern recognition.
原文链接
分享
引用
