Automated valet parking services provide great potential to increase the attractiveness of electric vehicles by mitigating their two main current deficiencies: reduced driving ranges and prolonged refueling times. The European research project V-Charge aims at providing this service on designated parking lots using close-to-market sensors only. For this purpose the project developed a prototype capable of performing fully automated navigation in mixed traffic on designated parking lots and GPS-denied parking garages with cameras and ultrasonic sensors only. This paper summarizes the work of the project, comprising advances in network communication and parking space scheduling, multi-camera calibration, semantic mapping concepts, visual localization and motion planning. The project pushed visual localization, environment perception and automated parking to centimetre precision. The developed infrastructure-based camera calibration and semi-supervised semantic mapping concepts greatly reduce maintenance efforts. Results are presented from extensive month-long field tests.

Robots that use vision for localization need to handle environments that are subject to seasonal and structural change, and operate under changing lighting and weather conditions. We present a framework for lifelong localization and mapping designed to provide robust and metrically accurate online localization in these kinds of changing environments. Our system iterates between offline map building, map summary, and online localization. The offline mapping fuses data from multiple visually varied datasets, thus dealing with changing environments by incorporating new information. Before passing these data to the online localization system, the map is summarized, selecting only the landmarks that are deemed useful for localization. This Summary Map enables online localization that is accurate and robust to the variation of visual information in natural environments while still being computationally efficient. We present a number of summary policies for selecting useful features for localization from the multisession map, and we explore the tradeoff between localization performance and computational complexity. The system is evaluated on 77 recordings, with a total length of 30 kilometers, collected outdoors over 16 months. These datasets cover all seasons, various times of day, and changing weather such as sunshine, rain, fog, and snow. We show that it is possible to build consistent maps that span data collected over an entire year, and cover day-to-night transitions. Simple statistics computed on landmark observations are enough to produce a Summary Map that enables robust and accurate localization over a wide range of seasonal, lighting, and weather conditions.

We present a map architecture based on a relational database that helps tackle the challenge of lifelong visuallocalization and mapping. The proposed design is rooted in a set of use-cases that describe the processes necessary for creating, using and analyzing visual maps. Our database and software architecture effectively expresses the requiredinteractions between map elements, such as visual frames generated by multi-camera systems. One of the major strengths of the proposed system is the ease of formulating pertinent and novel queries. We show how these queries can help us gaina better intuition about the map contents, taking into account complex data associations, even as session upon session is added to the map. Furthermore, we demonstrate how referential integrity checks, rollbacks and similar features of relational database management systems are beneficial for building long-term maps. Based on our experience with the proposed system during one year of intensive data collection and analysis, we discuss key lessons learned and indicate directions for evolving its design. These lessons show the importance of using higher relational normal forms to make the database schema even more useful for querying, as well as the need for a distributed, versioned system.

With the introduction of surround view cameras in modern vehicles and the possibility of calculating dense motion fields in real-time from a moving camera a detailed 3D reconstruction of the static environment is possible (structure-from-motion). Beside the necessity of a motion field between two image frames the task of triangulating those individual 2D point matches to 3D points in the world becomes non real-time on modern CPUs when to be repeated for all image points. In this work we evaluate different approaches to the 3D triangulation optimization problem in a typical structure-from-motion processing chain for an efficient implementation in hardware. An architecture for solving this problem using linear triangulation with an inhomogeneous solution to the equation system is proposed. We evaluate our implementation using FPGAs against a software-implementation with synthetic datasets and from low-speed parking area scenes for numerical accuracy and real-time capabilities. In addition the proposed fixed-point arithmetic implementation is compared against an implementation using floating-point units.

Future requirements for drastic reduction of CO2 production and energy consumption will lead to significant changes in the way we see mobility in the years to come. However, the automotive industry has identified significant barriers to the adoption of electric vehicles, including reduced driving range and greatly increased refueling times. Automated cars have the potential to reduce the environmental impact of driving, and increase the safety of motor vehicle travel. The current state-of-the-art in vehicle automation requires a suite of expensive sensors. While the cost of these sensors is decreasing, integrating them into electric cars will increase the price and represent another barrier to adoption. The V-Charge Project, funded by the European Commission, seeks to address these problems simultaneously by developing an electric automated car, outfitted with close-to-market sensors, which is able to automate valet parking and recharging for integration into a future transportation system. The final goal is the demonstration of a fully operational system including automated navigation and parking. This paper presents an overview of the V-Charge system, from the platform setup to the mapping, perception, and planning sub-systems.

The European V-Charge project seeks to develop fully automated valet parking and charging of electric vehicles using only low-cost sensors. One of the challenges is to implement robust visual localization using only cameras and stock vehicle sensors. We integrated four monocular, wide-angle, fisheye cameras on a consumer car and implemented a mapping and localization pipeline. Visual features and odometry are combined to build and localize against a keyframe-based three dimensional map. We report results for the first stage of the project, based on two months worth of data acquired under varying conditions, with the objective of localizing against a map created offline.

We present a complete processing chain for computing 2D occupancy grids from image sequences. A multi layer grid is introduced which serves several purposes. First the 3D points reconstructed from the images are distributed onto the underlying grid. Thereafter a virtual measurement is computed for each cell thus reducing computational complexity and rejecting potential outliers. Subsequently a height profile is updated from which the current measurement is partitioned into ground and obstacle pixels. Different height profile update strategies are tested and compared yielding a stable height profile estimation. Lastly the occupancy layer of the grid is updated. To asses the algorithm we evaluate it quantitatively by comparing the output of it to ground truth data illustrating its accuracy. We show the applicability of the algorithm by using both, dense stereo reconstructed and sparse structure and motion points. The algorithm was implemented and run online on one of our test vehicles in real time.

This paper presents a Structure from Motion based algorithm that enables perception of static vehicle surroundings using a single forward-looking camera. The classical computer vision methods are thoroughly examined and adjusted in order to ensure their suitability to the automotive environments. The focus in this work is placed on the analysis of the accuracy of the developed algorithms, which is performed based on both simulated and real data.