Face detection is a fundamental task for various computer vision applications. With the rapid improvement of computing power, most face detection algorithms adopt complex deep neural net architectures to increase accuracy. However, such algorithms are difficult to be applied on the mobile platform with restrained computing resources. To this end, this paper introduces a novel neuromorphic vision based single-shot driver face detection method, named NeuroDFD to handle driver face detection problem. Different from traditional CMOS camera, the neuromorphic vision sensor captures pixel level brightness changes signal and output them asynchronously in the form of events. It has the characteristics of high dynamic range and low data redundancy, which helps to meet the challenges of in-cabin lighting conditions and limited computing resources of edge devices. Based on the unique output, we first construct event representation by discretizing the time domain and present light weight translation-invariant backbone to extract multi-scale features. Then, we propose shift FPN and shift context module to promote the spatial-temporal features extraction with limited computation cost. Extensive experiments prove that the NeuroDFD can achieve remarkable detection performance with high efficiency.

Estimating the number of targets and their states is an important aspect of sensor fusion. In some applications, like autonomous driving, multiple measurements stem from extended targets because of multiple reflections from the target’s shape when using high resolution sensors like LiDAR or Radar. Multi-target tracking techniques using point based target assumptions are generally not suitable for these types of sensor measurements. In the last years, a number of techniques have been introduced which use a known shape or estimate the shape to retrieve the position of the object. In this paper we will introduce a novel approach without knowing/estimating the shape but using all the available information by fusing the measurements from one object with a conservative fusion technique based on the Weighted Exponential Product rule. The results show that we obtain similar performance to state-of-the-art approaches in our simulations.

This paper proposes a framework for uncertainty prediction in complex fusion networks, where signals become available sporadically. Assuming there is no information of the sensor characteristics available, a surrogated model of the sensor uncertainty is yielded directly from data through artificial neural networks. The strategy developed is applied to autonomous vehicle localization through odometry sensors (speed and orientation), so as to determine the location uncertainty in the trajectory. The results obtained allow for fusion of autonomous vehicle location measurements, and effective correction of the accumulated odometry error in most scenarios. The neural networks applicability and generalization capacity are proven, evidencing the suitability of the presented methodology for uncertainty estimation in non-linear and intractable processes.

Estimating the position, velocity and orientation of a vehicle is an extremely important aspect of highly assisted and autonomous driving scenarios. As a result of decades of research into this topic, there exist many tracking algorithms, each with different operating principles driven from different statistical frameworks. However, due to the complexity of the applications with which they are applied to, no algorithm has sufficient generality to be applied in all circumstances. While the topic of assessing the performance of algorithms has been investigated in the past, there exists no standardized framework for comparing the performance of different algorithms. In this paper we introduce an information theoretic framework which uses the Kullback Leibler Divergence to consider the relative information gain between different fusion algorithms. This framework is independent of the sensor systems and trajectories and considers only the technical operation of the algorithms. The results presented in this paper illustrate the utility of this approach and provide valuable insight for the development of algorithmic methodologies for real world vehicle dynamics estimation.

This paper presents a novel and an improved approach for estimating the position of a vehicle using vehicleinfrastructure cooperative localization.In our previous work we presented a Factor Graph based solution which added the topology (inter-vehicle distance) as a constraint while localizing the vehicle using data from sensors from both inside and outside the vehicle.This paper extends the work by reducing the error in calculating the precision of the position by almost 27% in the best case and lowering the computational time by at least 50% over our previously proposed solution.This is achieved by modifying current topology constraints to be also dependent on the previous state estimate.The proposed solution remains scalable for many vehicles without increasing the execution complexity.Finally, simulations indicate that incorporating the new topology information via Factor Graphs can improve performance over the traditional, state of the art, Kalman Filter approach.

Autonomous vehicles operating in dynamic environments rely on precise localization. In this paper we present a novel approach for cooperative localization of vehicular systems and an infrastructure RADAR which is resilient against outliers generated from the RADAR. The problem of cooperative localization is represented as a factor graph, where interrelated topologies (including that of outliers) are added as constraint factor between vehicle states. Corresponding probabilities for multiple topologies between states of the two vehicles are calculated using the Probability Data Association Filter and assigned to the respective edges in the graph. Simulation results indicate that this technique has significant benefits in the context of improving the resilience against outliers while optimizing joint state estimates. The methodology presented in this paper has the potential to provide a robust and flexible framework for cooperative localization in the presence of clutter, obscuration and targets entering and leaving the field of view.

Within the complex driving environment, progress in autonomous vehicles is supported by advances in sensing and data fusion. Safe and robust autonomous driving can only be guaranteed provided that vehicles and infrastructure are fully aware of the driving scenario. This paper proposes a methodology for feature uncertainty prediction for sensor fusion by generating neural network surrogate models directly from data. This technique is particularly applied to vehicle location through odometry measurements, vehicle speed and orientation, to estimate the location uncertainty at any point along the trajectory. Neural networks are shown to be a suitable modeling technique, presenting good generalization capability and robust results.

Data or measurement-to-track association is an integral and expensive part of any solution performing multi-target multi-sensor Cooperative Localization (CL) for better state estimation. Various performance evaluations have been performed between various state-of-the-art solutions, but they have been often limited within same family of algorithms. However, there exist solutions which avoid the task of data association to perform the CL in a multi-target multi-sensor environment. Factor Graphs using Symmetric Measurement Equations (SMEs) factor is one such solution. In this paper we compare and contrast the state estimation using state-of-the-art Random Finite Set (RFS) approach and using a Factor Graph solution with SMEs. For a RFS we use multi-sensor multi-object with the Generalized Labeled Multi-Bernoulli (GLMB) Filter. These two solution use conceptually different approaches, GLMB Filter solves the data association implicitly, but Factor Graph based solution avoids the task altogether. Simulations present an interesting results where for simple scenarios implemented GLMB filter performs efficiently. But the performance of GLMB Filter degrades faster than Factor Graphs using SMEs when the error in the sensors increase.

Precise localization is a key requirement for the success of highly assisted or autonomous vehicles. The diminishing cost of hardware has resulted in a proliferation of the number of sensors in the environment. Cooperative localization (CL) presents itself as a feasible and effective solution for localizing the ego-vehicle and its neighboring vehicles. However, one of the major challenges to fully realize the effective use of infrastructure sensors for jointly estimating the state of a vehicle in cooperative vehicle-infrastructure localization is an effective data association. In this paper, we propose a method which implements symmetric measurement equations within factor graphs in order to overcome the data association challenge with a reduced bandwidth overhead. Simulated results demonstrate the benefits of the proposed approach in comparison with our previously proposed approach of topology factors.

This paper describes a new approach for cooperative localization by using both internal and external sensors. In contrast to the state-of-the-art methods, the proposed approach analyses the statistical properties of the systematic error during the transformation phase. A factor graph is formulated which jointly estimates both the biases and the locations. The proposed approach is evaluated by using simulated data from odometry, GPS and radar measurements. The experiment demonstrates excellent performance of the proposed approach in comparison to traditional techniques.

Precise and accurate localization is important for safe autonomous driving. Given a traffic scenario which has multiple vehicles equipped with internal sensors for self-localization, and external sensors from the infrastructure for vehicle localization, vehicle-infrastructure communication can be used to improve the accuracy and precision of localization. However, as the number of vehicles in a scenario increases, associating measurement data with the correct source becomes increasingly challenging. We propose a solution utilizing the symmetric measurement equation filter (SME) for cooperative localization to address data association issue, as it does not require an enumeration of measurement-to-target associations. The principal idea is to define a symmetrical transformation which maps measurements to a homogeneous function, thereby effectively addressing several challenges in vehicle-infrastructure scenarios such as data association, bandwidth limitations and registration/configuration of the external sensor. To the best of our knowledge, the proposed solution is among the first to address all these issues of cooperative localization simultaneously, by utilizing the topology information of the vehicles.

Highly assisted and Autonomous Driving is dependent on the accurate localization of both the vehicle and other targets within the environment. With increasing traffic on roads and wider proliferation of low cost sensors, a vehicle-infrastructure cooperative localization scenario can provide improved performance over traditional mono-platform localization. The paper highlights the various challenges in the process and proposes a solution based on Factor Graphs which utilizes the concept of topology of vehicles. A Factor Graph represents probabilistic graphical model as a bipartite graph. It is used to add the inter-vehicle distance as constraints while localizing the vehicle. The proposed solution is easily scalable for many vehicles without increasing the execution complexity. Finally simulation indicates that incorporating the topology information as a state estimate can improve performance over the traditional Kalman Filter approach.

A key assumption of distributed data fusion is that individual nodes have no knowledge of the global network topology and use only information which is available locally. This paper considers the weighted exponential product (WEP) rule as a methodology for conservatively fusing estimates with an unknown degree of correlation between them. We provide a preliminary investigation into how the methodology for selecting the mixing parameter can be used to minimize the information loss in the fused covariance as opposed to reducing the Shannon entropy, and hence maximize the information of the fused covariance. Our results suggest that selecting a mixing parameter which minimizes the information loss ensures that information which is exclusive to the estimates from one source is not lost during the fusion process. These results indicate that minimizing the information loss provides a robust technique for selecting the mixing parameter in WEP fusion.

In this paper we investigate the feasibility for using a Synthetic Aperture Radar (SAR) to detect radar scatterers in support of advanced driver assistance systems. Specifically, we consider the detection of radar scatterers physically embedded into lane and carriageway boundaries similar to way optical retroreflectors (cats eyes) are used in present infrastructure. We use simulations to generate high resolution SAR images for detecting and localizing radar scatterers. The simulated results presented here highlight the feasibility of the technique and provide a platform for further investigation. This paper facilitates the realization of the role of modified infrastructure for improving the sensing capability of highly assisted and autonomous vehicles.

In this paper, we propose a Bernoulli filter for estimating a vehicle’s trajectory under random finite set (RFS) framework. In contrast to other approaches, ego-motion vector is considered as the state of an extended target while the features are considered as multiple measurements that originated from the target. The Bernoulli filter estimates the state of the extended target instead of tracking individual features, which presents a recursive filtering framework in the presence of high association uncertainty. Experimental results illustrate that the proposed approach exhibits good robustness under real traffic scenarios.

We describe in this paper our gesture detection and recognition system for the 2014 ChaLearn Looking at People (Track 3: Gesture Recognition) organized by ChaLearn in conjunction with the ECCV 2014 conference. The competition's task was to learn a vacabulary of 20 types of Italian gestures and detect them in sequences. Our system adopts a multi-modality approach for detecting as well as recognizing the gestures. The goal of our approach is to identify semantically meaningful contents from dense sampling spatio-temporal feature space for gesture recognition. To achieve this, we develop three concepts under the random forest framework: un-supervision; discrimination; and randomization. Un-supervision learns spatio-temporal features from two channels (grayscale and depth) of RGB-D video in an unsupervised way. Discrimination extracts the information in dense sampling spatio-temporal space effectively. Randomization explores the dense sampling spatio-temporal feature space efficiently. An evaluation of our approach shows that we achieve a mean Jaccard Index of 0.6489, and a mean average accuracy of 90.3% over the test dataset.

Precise and accurate localization is important for safe autonomous driving. Given a traffic scenario with multiple vehicles equipped with proprioceptive sensors for self-localization and infrastructure equipped with exteroceptive sensors for car detection, vehicle-infrastructure communication can be used to improve the localization. However as the number of vehicles in a scenario increases, data association becomes increasingly challenging. We propose a solution utilizing the symmetric measurement equation filter (SME) for cooperative localization to address data association issues, as it does not require an enumeration of measurement-to-target associations. The key idea is to define a symmetric transformation which maps position measurements to a homogeneous function, thereby effectively addressing several challenges in vehicle-infrastructure scenarios such as bandwidth limitations, data association challenges and especially the configuration of the exteroceptive sensor. The approach works well even in the case that the location and orientation of the exteroceptive sensor are unknown. To the best of our knowledge, our proposed solution is among the first to address all these challenges of cooperative localization simultaneously, by utilizing the topology information of the vehicles. A comparative study based on simulations demonstrates the reliability and the feasibility of the proposed approach in 2D coordinates.

Previous work on 3D action recognition has focused on using hand-designed features, either from depth videos or 2D videos. In this work, we present an effective way to combine unsupervised feature learning with discriminative feature mining. Unsupervised feature learning allows us to extract spatio-temporal features from unlabeled video data. With this, we can avoid the cumbersome process of designing feature extraction by hand. We propose an ensemble approach using a discriminative learning algorithm, where each base learner is a discriminative multi-kernel-learning classifier, trained to learn an optimal combination of joint-based features. Our evaluation includes a comparison to state-of-the-art methods on the MSRAction 3D dataset, where our method, abbreviated EnMkl, outperforms earlier methods. Furthermore, we analyze the efficiency of our approach in a 3D action recognition system.

Human action recognition based on joints is a challenging task. The 3D positions of the tracked joints are very noisy if occlusions occur, which increases the intra-class variations in the actions. In this paper, we propose a novel approach to recognize human actions with weighted joint-based features. Previous work has focused on hand-tuned joint-based features, which are difficult and time-consuming to be extended to other modalities. In contrast, we compute the joint-based features using an unsupervised learning approach. To capture the intra-class variance, a multiple kernel learning approach is employed to learn the skeleton structure that combine these joints-base features. We test our algorithm on action application using Microsoft Research Action3D (MSRAction3D) dataset. Experimental evaluation shows that the proposed approach outperforms state-of-the-art action recognition algorithms on depth videos.

In this paper, we consider scenarios, requirements, and architectures for future connected vehicles. Regarding scenarios, we discuss new features like autonomous driving, which have very strong requirements on sensor data fusion, real-time processing, and the complexity of the decision making process. We also address a major trend in automotive architectures, which is the aggregation of computing inside the vehicle. Furthermore, we consider connected vehicles, where functionality can be taken over by the cloud. This leads to new challenges with respect to allocation of real-time data processing and control as well as the distribution of the strategy layer across the tiers. To address the challenges of these novel scenarios, we propose a 3-tier architecture for future connected vehicles.