There is evidence that shows that the majority of all vehicle accidents are caused by human error. This has been the major motivation for advancements in Advanced Driving Assistance Systems (ADAS). A driver's gaze can provide valuable information about the focus and intention of a driver. Therefore, determining the degree of driver awareness and scene perception, and predicting driver intentions can be valuable in the next generation of ADAS. To this end, we have instrumented a vehicle with a front-facing stereoscopic vision system on the roof of the vehicle and a camera gaze tracking system pointing toward the driver's face. These are two completely different systems with dissimilar sensing modalities. Data is collected separately from these systems and when calibrated can be used to estimate the Point-of-Gaze (PoG) of the driver. We present an efficient approach for the cross-calibration of the gaze tracker with the stereoscopic vision system. The experimental results show that our proposed cross-calibration technique obtains promising results for estimating Point-of-Gaze (PoG).

A driver’s actions and intent can be factors in enabling advance driver assistance systems (ADASs) to assist drivers and avoid accidents. A driver’s gaze can provide insight into the driver’s intent or awareness of situations. Knowing that a driver gazed at a traffic sign or missed a traffic could provide indications of whether the driver is alert to impending changes in the driving environment, such as curves and stop signs. For ADASs to determine the importance of a driver seeing or missing a sign, it is important to understand the driving environment and situation. A first step is to understand what signs drivers do see or miss while driving. This contribution presents the results of analyzing driving sequences to assess traffic signs that drivers may or may not have gazed upon. The results suggest that drivers may miss 20% of traffic signs though the percentage varies depending on the type of sign. The analysis uses image sequences of the driving environment and gazes data captured during driving. The methods used in our analysis included determining whether a driver’s gaze has fallen on the image of a traffic sign or not and subsequently determining signs missed during driving. The methods presented can be useful in other scenarios involving the analysis of driver gaze and have implications for the design of future ADASs and for understanding of driver gaze and awareness.

The direction of a driver’s visual attention plays a crucial role in the context of Advanced Driver Assistance Systems (ADASs) and semi-autonomous driving. The way a driver monitors traffic scene objects partially indicates the level of driver awareness. We propose an analytical method to estimate a driver’s average traffic scene attention based on the attentional visual field of the driver in urban and suburban areas. Three metrics are proposed to estimate a driver’s average attention. Our model is capable of identifying driver attention with respect to traffic objects including vehicles, traffic lights, traffic signs, and pedestrians within the attentional visual field of the driver at any moment while in the act of driving.

Traffic object detection and recognition systems play an essential role in Advanced Driver Assistance Systems (ADAS) and Autonomous Vehicles (AV). In this research, we focus on four important classes of traffic objects: traffic signs, road vehicles, pedestrians, and traffic lights. We first review the major traditional machine learning and deep learning methods that have been used in the literature to detect and recognize these objects. We provide a vision-based framework that detects and recognizes traffic objects inside and outside the attentional visual area of drivers. This approach uses the driver 3D absolute coordinates of the gaze point obtained by the combined, cross-calibrated use of a front-view stereo imaging system and a non-contact 3D gaze tracker. A combination of multi-scale HOG-SVM and Faster R-CNN-based models are utilized in the detection stage. The recognition stage is performed with a ResNet-101 network to verify sets of generated hypotheses. We applied our approach on real data collected during drives in an urban environment with the RoadLAB instrumented vehicle. Our framework achieved 91% of correct object detections and provided promising results in the object recognition stage.

This study aims to analyze driver cephalo-ocular behaviour features and road vanishing points with respect to vehicle speed in urban and suburban areas using data obtained from an instrumented vehicle’s eye tracker. This study utilizes two models for driver gaze estimation. The first model estimates the 3D point of the driver’s gaze in absolute coordinates obtained through the combined use of a forward stereo vision system and an eye-gaze tracker system. The second approach uses a stochastic model, known as Gaussian Process Regression (GPR), that estimates the most probable gaze direction given head pose. We evaluated models on real data gathered in an urban and suburban environment with the RoadLAB experimental vehicle.

Road lane detection systems play a crucial role in the context of Advanced Driver Assistance Systems (ADASs) and autonomous driving. Such systems can lessen road accidents and increase driving safety by alerting the driver in risky traffic situations. Additionally, the detection of ego lanes with their left and right boundaries along with the recognition of their types is of great importance as they provide contextual information. Lane detection is a challenging problem since road conditions and illumination vary while driving. In this contribution, we investigate the use of a CNN-based regression method for detecting ego lane boundaries. After the lane detection stage, following a projective transformation, the classification stage is performed with a RseNet101 network to verify the detected lanes or a possible road boundary. We applied our framework to real images collected during drives in an urban area with the RoadLAB instrumented vehicle. Our experimental results show that our approach achieved promising results in the detection stage with an accuracy of 94:52% in the lane classification stage.

The driving environment is a complex dynamic scene in which a driver’s eye fixation interacts with traffic scene objects to protect the driver from dangerous situations. Prediction of a driver’s eye fixation plays a crucial role in Advanced Driving Assistance Systems (ADAS) and autonomous vehicles. However, currently, no computational framework has been introduced to combine the bottom-up saliency map with the driver’s head pose and gaze direction to estimate a driver’s eye fixation. In this work, we first propose convolution neural networks to predict the potential saliency regions in the driving environment, and then use the probability of the driver gaze direction, given head pose as a top-down factor. We evaluate our model on real data gathered during drives in an urban and suburban environment with an experimental vehicle. Our analyses show promising

The direction of a vehicle driver's visual attention plays an essential role in the research on Advanced Driving Assistance Systems (ADAS) and autonomous vehicles. How a driver monitors the surrounding environment is at least partially descriptive of the driver's situational awareness. While driver gaze is not explicitly related to head pose due to the interplay between head and eye movements, it may still provide an approximation of the visual attention that is sufficiently accurate for many applications. In this research, we propose a probabilistic method for describing the visual attention of drivers. This method applies a Gaussian Process Regression (GPR) technique that estimates the probability of the driver gaze direction, given head pose. We evaluate our model on real data collected during drives with an experimental vehicle in urban and suburban areas. Our experimental results show that 82.5% of drivers' gaze lies within the 95% confidence interval predicted by our framework.

This work introduces and evaluates a model for predicting driver behaviour, namely turns or proceeding straight, at traffic light intersections from driver three-dimensional gaze data and traffic light recognition. Based on vehicular data, this work relates the traffic light position, the driver's gaze, head movement, and distance from the centre of the traffic light to build a model of driver behaviour. The model can be used to predict the expected driver manoeuvre 3 to 4 s prior to arrival at the intersection. As part of this study, a framework for driving scene understanding based on driver gaze is presented. The outcomes of this study indicate that this deep learning framework for measuring, accumulating and validating different driving actions may be useful in developing models for predicting driver intent before intersections and perhaps in other key-driving situations. Such models are an essential part of advanced driving assistance systems that help drivers in the execution of manoeuvres.

Driver maneuver prediction is of great importance in designing a modern Advanced Driver Assistance System (ADAS). Such predictions can improve driving safety by alerting the driver to the danger of unsafe or risky traffic situations. In this research, we developed a model to predict driver maneuvers, including left/right lane changes, left/right turns and driving straight forward 3.6 seconds on average before they occur in real time. For this, we propose a deep learning method based on Long Short-Term Memory (LSTM) which utilizes data on the driver's gaze and head position as well as vehicle dynamics data. We applied our approach on real data collected during drives in an urban environment in an instrumented vehicle. In comparison with previous IOHMM techniques that predicted three maneuvers including left/right turns and driving straight, our prediction model is able to anticipate two more maneuvers. In addition to this, our experimental results show that our model using identical dataset improved F1 score by 4% and increased to 84%.

Vehicle License Plate Detection and Recognition has become critical to traffic, security and surveillance applications. This contribution aims to implement and evaluate different techniques for License Plate Detection and Recognition in order to improve their accuracy. This work addresses various problems in detection such as adverse weather, illumination change and poor quality of captured images. After detecting the license plate location in an image the next challenge is to recognize each letter and digit. In this work three different approaches have been investigated to find which one performs best. Here, characters are classified through template matching, multi-class SVM, and convolutional neural network. The performance was measured empirically, with 36 classes each containing 400 images per class used for training and testing. For each algorithm empirical accuracy was assessed.

We report results from the preliminary trials of Colibri, a dedicated fast-photometry array for the detection of small Kuiper Belt objects (KBOs) through serendipitous stellar occultations. Colibri's novel data processing pipeline analyzed 4000 star hours with two overlapping-field EMCCD cameras, detecting no KBOs and one false positive occultation event in a high ecliptic latitude field. No occultations would be expected at these latitudes, allowing these results to provide a control sample for the upcoming main Colibri campaign. The empirical false positive rate found by the processing pipeline is consistent with the 0.002% simulation-determined false positive rate. We also describe Colibri's software design, kernel sets for modeling stellar occultations, and method for retrieving occultation parameters from noisy diffraction curves. Colibri's main campaign will begin in mid-2018, operating at a 40 Hz sampling rate.

Communication for cooperative traffic management increases convenience and efficiency in driving. Intelligent vehicles can collect information about the driving environment, the driver situation and, more importantly, information on other vehicles. While giving this information to the driver can be useful, there is also the possibility of presenting the driver with too much information. Existing vehicles already have some mechanisms to take certain actions if the driver fails to act. Future vehicles will need more complex decision-making modules which receive the raw data from all available sources, process this data and inform the driver about existing or impending situations and suggest, or even take actions. In this work, we explore the use of a decision-making module for accident situations that processes information from VANET communication and advises the driver based on the situation. Our decision-making algorithm is a simple and effective algorithm that can be implemented in each vehicle to assist the driver in situations where rerouting to avoid traffic congestion caused by an accident may be necessary. Our approach estimates accident duration and provides a rerouting for upcoming traffic. Our algorithm has been implemented and evaluated within an elegant traffic simulation system where both city and highway environments are considered, along with various proportions of V2V-equipped vehicles. Overall, the results indicate that using our decision-making module in vehicles shows great potential for improving performance of vehicular systems by reducing both travel and wait times and providing more accurate information on surrounding environments.

Traffic sign detection and recognition systems are essential components of Advanced Driver Assistance Systems and self-driving vehicles. In this contribution we present a vision-based framework which detects and recognizes traffic signs inside the attentional visual field of drivers. This technique takes advantage of the driver 3D absolute gaze point obtained through the combined use of a front-view stereo imaging system and a non-contact 3D gaze tracker. We used a linear Support Vector Machine as a classifier and a Histogram of Oriented Gradient as features for detection. Recognition is performed by using Scale Invariant Feature Transforms and color information. Our technique detects and recognizes signs which are in the field of view of the driver and also provides indication when one or more signs have been missed by the driver.

We present a method for filtering noisy point clouds, specifically those constructed from merged depth maps as obtained from a range scanner or multiple view stereo (MVS), applying techniques that have previously been used in finding outliers in clustered data, but not in MVS or range scanning. We estimate the probability density function (PDF) over the space of observed points via a technique called kernel density estimation. We utilize Mahalanobis distance and a variable bandwidth for weighting kernels accordingly, based on the nature of neighbouring points. Further, we incorporate a distance metric called the Reachability Distance that, as we show in our results, gives better discrimination than a classical Mahalanobis distancebased metric. With the addition of this nearest neighbour metric, we can produce results that are ready for meshing without any post-processing of the cloud. We mesh our filtered point clouds using a traditional surface fitting technique that is unequipped to deal with noise to demonstrate the efficacy of our method.

Driving Assistance Systems increase safety and provide a more enjoyable driving experience. Among the objectives motivating these technologies rests the idea of predicting driver intent within the next few seconds, in order to avoid potentially dangerous manoeuvres. In this work, we develop a model of driver behaviour for turn manoeuvres that we then apply to anticipate the most likely turn manoeuvre a driver will effect a few seconds ahead of time. We demonstrate that cephalo-ocular behaviour such as variations in gaze direction and head pose play an important role in the prediction of driver-initiated manoeuvres. We tested our approach on a diverse driving data set recorded with an instrumented vehicle in the urban area of London, ON, Canada. Experiments show that our approach predicts turn manoeuvres 3.8 seconds before they occur with an accuracy over 80% in real-time.

The finite element method (FEM) has very broad applications in a lot of research areas, and isogeometric analysis (IGA) is a new advancement based on FEM to integrate design with analysis. This chapter reviews the basic algorithm of finite element analysis (FEA) and its new developments, including IGA, extended FEM and immersed FEM. As a popular and powerful numerical method to solve partial differential equations over complex domains, FEM has been developed rapidly and used in many research areas including computational medicine, biology and engineering. The FEM is a general technique to solve boundary value problems with uniformly and non-uniformly spaced grids or meshes. In the implementation, the element stiffness matrix and element load vector are computed element by element, and then assembled together into the global stiffness matrix and global load vector. FEA has been applied …

Vehicle self-localization is an important aspect of intelligent transportation systems. Global Positioning Systems (GPS) which provide vehicle localization information play an important role in these systems. However, GPS is challenged in urban environments where satellite visibility and multipath situations are unavoidable. In this contribution we propose a method by which vehicular speed and a map-based lane detection process are called upon to improve the positional accuracy of GPS. Experimental results with urban driving sequences demonstrate that our approach significantly improves the accuracy of positioning the vehicle as compared with systems solely relying on GPS.

This work aims to propose and validate a framework for tumour volume auto-segmentation based on ground-truth estimates derived from multi-physician input contours to expedite 4D-CT based lung tumour volume delineation. 4D-CT datasets of ten non-small cell lung cancer (NSCLC) patients were manually segmented by 6 physicians. Multi-expert ground truth (GT) estimates were constructed using the STAPLE algorithm for the gross tumour volume (GTV) on all respiratory phases. Next, using a deformable model-based method, multi-expert GT on each individual phase of the 4D-CT dataset was propagated to all other phases providing auto-segmented GTVs and motion encompassing internal gross target volumes (IGTVs) based on GT estimates (STAPLE) from each respiratory phase of the 4D-CT dataset. Accuracy assessment of auto-segmentation employed graph cuts for 3D-shape reconstruction and point-set registration-based analysis yielding volumetric and distance-based measures. STAPLE-based auto-segmented GTV accuracy ranged from (81.51  ±  1.92) to (97.27  ±  0.28)% volumetric overlap of the estimated ground truth. IGTV auto-segmentation showed significantly improved accuracies with reduced variance for all patients ranging from 90.87 to 98.57% volumetric overlap of the ground truth volume. Additional metrics supported these observations with statistical significance. Accuracy of auto-segmentation was shown to be largely independent of selection of the initial propagation phase. IGTV construction based on auto-segmented GTVs within the 4D-CT dataset provided accurate and reliable target volumes compared to manual segmentation-based GT estimates. While inter-/intra-observer effects were largely mitigated, the proposed segmentation workflow is more complex than that of current clinical practice and requires further development.

We present a global algorithm for drift free alignment of multiple range scans of "thin" data into a single point cloud that is suitable for further processing, such as triangular meshing and volume calculation. We consider two sets of non-rigid data: synthetic vascular data and real Arabidopsis plant data. Our method builds on the coherent point drift algorithm, and aligns multiple point clouds into a single 3D point cloud. The plant data was acquired in a growth chamber, where the fan caused jittering in both the branch and leaf data. For each scan, we construct a target scan from the cancroids of its Mutual Nearest Neighbours (MNN) in all other scans and iteratively register to this, as opposed to registering pair wise scans sequentially. We have have adapted MNN for use in non-rigid scenarios, producing a method that will will not degrade as more scans are registered, and produces better results than sequential pair wise registration.