Reliable pedestrian detection represents a crucial step towards automated driving systems. However, the current performance benchmarks exhibit weaknesses. The currently applied metrics for various subsets of a validation dataset prohibit a realistic performance evaluation of a DNN for pedestrian detection. As image segmentation supplies fine-grained information about a street scene, it can serve as a starting point to automatically distinguish between different types of errors during the evaluation of a pedestrian detector. In this work, eight different error categories for pedestrian detection are proposed and new metrics are proposed for performance comparison along these error categories. We use the new metrics to compare various backbones for a simplified version of the APD, and show a more fine-grained and robust way to compare models with each other especially in terms of safety-critical performance. We achieve SOTA on CityPersons-reasonable (without extra training data) by using a rather simple architecture.

Basis of a robust safety strategy for an automated driving function based on neural networks is a detailed description of its input domain, i.e. a description of the environment, in which the function is used. This is required to describe its functional system boundaries and to perform a comprehensive safety analysis. Moreover, it allows to tailor datasets specifically designed for safety related validation tests. Ontologies fulfill the task to gather expert knowledge and model information to enable computer aided processing, while using a notion understandable for humans. In this contribution, we propose a methodology for domain analysis to build up an ontology for perception of autonomous vehicles including characteristic features that become important when dealing with neural networks. Additionally, the method is demonstrated by the creation of a synthetic test dataset for an Euro NCAP-like use case.

The automation of driving systems relies on proof of the correct functioning of perception. Arguing the safety of deep neural networks (DNNs) must involve quantifiable evidence. Currently, the application of DNNs suffers from an incomprehensible behavior. It is still an open question if post-hoc methods mitigate the safety concerns of trained DNNs. Our work proposes a method for inherently interpretable and concept-based pedestrian detection (CPD). CPD explicitly structures the latent space with concept vectors that learn features for body parts as predefined concepts. The distance-based clustering and separation of latent representations build an interpretable reasoning process. Hence, CPD predicts a body part segmentation based on distances of latent representations to concept vectors. A non-interpretable 2d bounding box prediction for pedestrians complements the segmentation. The proposed CPD generates additional information that can be of great value in a safety argumentation of a DNN for pedestrian detection. We report competitive performance for the task of pedestrian detection. Finally, CPD enables concept-based tests to quantify evidence of a safe perception in automated driving systems.

The use of deep neural networks (DNNs) in safety-critical applications like mobile health and autonomous driving is challenging due to numerous model-inherent shortcomings. These shortcomings are diverse and range from a lack of generalization over insufficient interpretability to problems with malicious inputs. Cyber-physical systems employing DNNs are therefore likely to suffer from safety concerns. In recent years, a zoo of state-of-the-art techniques aiming to address these safety concerns has emerged. This work provides a structured and broad overview of them. We first identify categories of insufficiencies to then describe research activities aiming at their detection, quantification, or mitigation. Our paper addresses both machine learning experts and safety engineers: The former ones might profit from the broad range of machine learning topics covered and discussions on limitations of recent methods. The latter ones might gain insights into the specifics of modern ML methods. We moreover hope that our contribution fuels discussions on desiderata for ML systems and strategies on how to propel existing approaches accordingly.

AI-based perception is a key factor towards the automation of driving systems. A conclusive safety argumentation must provide evidence for safe functioning. Existing safety standards are not suitable to deal with non-interpretable deep neural networks (DNN) learning from unstructured data. This work provides a proof of concept for a comprehensible requirements analysis based on an interpretable DNN. Recent work on interpretability motivates to rethink software considerations of safety standards. We describe the application of established considerations to DNNs by integrating interpretability and identifying artifacts. DNN artifacts result from a meaningful decomposition of requirements and adaptions of the perception pipeline. To prove our concept, we propose an interpretable method for the center, scale and prototype prediction (CSPP) that learns an explicitly structured latent space. The interpretability-based requirements analysis of CSPP is completed by tracing artifacts and source code to decomposed requirements. Finally, qualitative post-hoc evaluations provide evidence for the fulfillment of defined requirements for the latent space.