On the Road to Clarity: Exploring Explainable AI for World Models in a Driver Assistance System
arxiv(2024)
摘要
In Autonomous Driving (AD) transparency and safety are paramount, as mistakes
are costly. However, neural networks used in AD systems are generally
considered black boxes. As a countermeasure, we have methods of explainable AI
(XAI), such as feature relevance estimation and dimensionality reduction.
Coarse graining techniques can also help reduce dimensionality and find
interpretable global patterns. A specific coarse graining method is
Renormalization Groups from statistical physics. It has previously been applied
to Restricted Boltzmann Machines (RBMs) to interpret unsupervised learning. We
refine this technique by building a transparent backbone model for
convolutional variational autoencoders (VAE) that allows mapping latent values
to input features and has performance comparable to trained black box VAEs.
Moreover, we propose a custom feature map visualization technique to analyze
the internal convolutional layers in the VAE to explain internal causes of poor
reconstruction that may lead to dangerous traffic scenarios in AD applications.
In a second key contribution, we propose explanation and evaluation techniques
for the internal dynamics and feature relevance of prediction networks. We test
a long short-term memory (LSTM) network in the computer vision domain to
evaluate the predictability and in future applications potentially safety of
prediction models. We showcase our methods by analyzing a VAE-LSTM world model
that predicts pedestrian perception in an urban traffic situation.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要