An efficient transportation system leads to increased safety, better financial security, as well as a cleaner environment. We propose an approach for significantly improving efficiency in heterogeneous unregulated traffic environments. Existing intelligent transportation solutions for improving efficiency, however, apply only to traffic intersections with sparse traffic or traffic where drivers obey the regulations, or both. Our approach combines an auction mechanism with an optimization-based trajectory planner. The auction mechanism generates a fair priority entrance sequence for heterogeneous agents while the trajectory planner computes velocity controls in real-time, taking less than 10 milliseconds even in high-density traffic with over 10, 000 vehicles per hour. Compared to fully optimization-based methods, it operates 100x faster while ensuring fairness, safety, and efficiency. Tested on the SUMO simulator, our algorithm improves throughput by at least 25%, reduces the time to reach the goal by at least 70%, and decreases fuel consumption by 50% compared to approaches using traffic lights and stop signs. Our approach is also unaffected by unbalanced traffic inflows, whereas some of the other baselines encountered a decrease in performance in unbalanced traffic inflow environments.

Better fuel efficiency leads to better financial security as well as a cleaner environment. We propose a novel approach for improving fuel efficiency in unstructured and unregulated traffic environments. Existing intelligent transportation solutions for improving fuel efficiency, however, apply only to traffic intersections with sparse traffic or traffic where drivers obey the regulations, or both. We propose GameOpt+, a novel hybrid approach for cooperative intersection control in dynamic, multi-lane, unsignalized intersections. GameOpt+ is a hybrid solution that combines an auction mechanism and an optimization-based trajectory planner. It generates a priority entrance sequence for each agent and computes velocity controls in real-time, taking less than 10 milliseconds even in high-density traffic with over 10,000 vehicles per hour. Compared to fully optimization-based methods, it operates 100 times faster while ensuring fairness, safety, and efficiency. Tested on the SUMO simulator, our algorithm improves throughput by at least 25 time to reach the goal by at least 70 compared to auction-based and signaled approaches using traffic lights and stop signs. GameOpt+ is also unaffected by unbalanced traffic inflows, whereas some of the other baselines encountered a decrease in performance in unbalanced traffic inflow environments.

Coordinating the flow of traffic through urban areas with multiple intersections is a complex problem whose solution has the potential to improve safety, increase throughput, and optimize energy efficiency. In addition to controlling traffic lights, the introduction of connected and automated vehicles (CAVs) offers opportunities in terms of additional sensing and actuation points within the traffic network. This paper proposes a centralized and a decentralized implementation for the joint coordination and control of both traffic signals and mixed traffic, including CAVs and human driven vehicles (HDVs), in a network of multiple connected traffic intersections. Mixed-integer linear programming (MILP) is used to compute safe control trajectories for both CAVs and traffic light signals, which minimize overall congestion and fuel consumption. Our approaches are validated using extensive traffic simulations on the SUMO platform and they are shown to provide improvements of around 32-60%, 90-96% and 40-60% in travel time, waiting time and fuel consumption, respectively, when compared to gap-based adaptive and timed traffic lights.

Safe overtaking, especially in a bidirectional mixed-traffic setting, remains a key challenge for Connected Autonomous Vehicles (CAVs). The presence of human-driven vehicles (HDVs), behavior unpredictability, and blind spots resulting from sensor occlusion make this a challenging control problem. To overcome these difficulties, we propose a cooperative communication-based approach that utilizes the information shared between CAVs to reduce the effects of sensor occlusion while benefiting from the local velocity prediction based on past tracking data. Our control framework aims to perform overtaking maneuvers with the objective of maximizing velocity while prioritizing safety and passenger comfort. Our method is also capable of reactively adjusting its plan to dynamic changes in the environment. The performance of the proposed approach is verified using realistic traffic simulations.

Traffic shock waves are well-known naturally occurring phenomena that lead to unnecessary congestion in highway networks. Introducing connected autonomous vehicles (CAVs) to highways of human-driven vehicles (HDVs) allows for the development of traffic control schemes that can mitigate the effects of the shock waves. In this work, we propose a shock wave detection algorithm based on communication between CAVs with local traffic information. The proposed methodology is suitable for multi-lane mixed traffic highways of arbitrary structure, i.e., it is not limited to closed-circuit ring roads. We show that the detection information can be used to design a class of proactive shock wave mitigating CAV controllers. The choice of the controller can depend on design parameters such as the aggressiveness of the driving behavior allowed. We also demonstrate the importance of the positioning of autonomous agents in multi-lane scenarios. The shock wave dissipation efficiency is evaluated on a three lane highway loop using realistic traffic simulations and low CAV penetration levels.

Traffic shock waves are a commonly occurring phenomena caused by the delays in reaction times of Human Driven Vehicles (HDVs) resulting in unnecessary congestion in highway networks. Application of a suitable moving bottleneck control using Connected Autonomous Vehicles (CAVs) can result in shock wave mitigation and smoothing of the traffic flow. This traffic control scheme is dependent on accurately predicting shock wave conditions while choosing the best control to apply for the observation available to the CAV. In this work, we propose the use of a multi-agent shared policy reinforcement learning algorithm which leverages communication between CAVs for improved observability of downstream traffic conditions. A key feature of this method is the ability to perform shock wave dissipation control without the need for global information and the applicability of this method to multi-lane mixed traffic highways of arbitrary structure. We use the shared-parameter Proximal Policy Optimization (PPO) reinforcement learning strategy for obtaining the controls for each CAV in the simulation. We also built a custom SUMO-Gym wrapper for the multi-lane highway simulation with custom designed observation space, action space and rewards for each agent. The shock wave dissipation efficiency is evaluated on a three lane circular highway loop using realistic traffic simulation software and low CAV penetration levels.

With the advent of autonomous vehicles on public roads imminent in the near future, special emphasis needs to be placed on addressing scenarios pertaining to mixed-traffic settings, comprised of human-driven and autonomous vehicles. In this paper, we address the problem of autonomous vehicle overtaking in a bidirectional mixed-traffic setting. We design a mixed-integer model predictive controller that maximizes the ego vehicle’s velocity while prioritizing safety and accounting for driver comfort. The proposed approach: (i) operates in a limited sensing range while accounting for occlusion; (ii) is able to retract the overtake decision through a receding horizon approach; (iii) is robust to the variations in sensory input and driving behaviors of external agents due to behavior-dependent safety margins; and (iv) reduces to a mixed-integer optimization problem with linear constraints, yielding low computational complexity. We demonstrate the behavior of the proposed approach in a realistic traffic simulation environment.

We propose GAMEOPT: a novel hybrid approach to cooperative intersection control for dynamic, multi-lane, unsignalized intersections. Safely navigating these complex and accident prone intersections requires simultaneous trajectory planning and negotiation among drivers. GAMEOPT is a hybrid formulation that first uses an auction mechanism to generate a priority entrance sequence for every agent, followed by an optimization-based trajectory planner that computes velocity controls that satisfy the priority sequence. This coupling op-erates at real-time speeds of less than 10 milliseconds in high density traffic of more than 10, 000 vehicles/hr, 100× faster than other fully optimization-based methods, while providing guarantees in terms of fairness, safety, and efficiency. Tested on the SUMO simulator, our algorithm improves throughput by at least 25%, time taken to reach the goal by 75%, and fuel consumption by 33% compared to auction-based approaches and signaled approaches using traffic-lights and stop signs.

We propose a reinforcement learning algorithm based on an adaptive state aggregation scheme defined by a progressively growing set of codevectors placed in the joint state-action space according to a maximum-entropy vector quantization scheme. The proposed algorithm constitutes a two-timescale stochastic approximation algorithm with: (a) a fast component that executes a temporal-difference learning algorithm, and (b) a slow component, based on an online deterministic annealing algorithm, that adaptively partitions the state-action space according to a dissimilarity measure that belongs to the family of Bregman divergences. The proposed online deterministic annealing algorithm is a competitive-learning neural network that shows robustness with respect to the initial conditions, requires minimal hyper-parameter tuning, and provides online control over the performance-complexity trade-off. We study the convergence properties of the proposed methodology and quantify its performance in simulated experiments. Finally, we show that the generated codevectors can be used as training samples for sparse and progressively more accurate Gaussian process regression.

Highway on-ramp merge junctions remain a major bottleneck in transportation networks. However, with the introduction of Connected Autonomous Vehicles (CAVs) with advanced sensing and communication capabilities modern algorithms can capitalize on the cooperation between vehicles. This paper enhances highway merging efficiency by optimally coordinating CAVs in order to maximize the flow of vehicles while satisfying all safety constraints. Focus is also placed on the effect of varying priorities of different vehicle classes in selecting the best merging sequence. Our algorithm is capable of real time operation through parallel computation, optimized merge sequence generation and management of the diverse needs of heterogeneous (multi-class) traffic. Results are verified through a realistic traffic simulation software.

In the presence of human driven vehicles (HDVs), traffic shock waves are a naturally occurring phenomena, which contributes to congestion and efficiency degradation in highway networks. The introduction of connected autonomous vehicles (CAVs) with advanced sensing, actuation and communication capabilities allow new approaches in control to be applied in order to solve the problem of shock waves. This work on shock wave dissipation, introduces a communication-based cooperative control method for CAVs in multi-lane highways in a mixed traffic setting. The method allows for proactive control application and exhibits good shock wave dissipation performance even with low CAV penetration levels. Results are verified on a three lane circular highway loop using realistic traffic simulation software.

. Abstract While reinforcement learning algorithms based on parametric models can deal with the curse of dimensionality, convergence properties can be difficult to establish and their performance in practice heavily depends on the choice of the basis functions. We propose a reinforcement learning algorithm based on an adaptive aggregation scheme defined by a progressively growing set of codevectors placed in the joint state-action space according to a maximum-entropy vector quantization scheme. The proposed algorithm can be used for robot control and constitutes a two-timescale stochastic approximation algorithm with: (a) a fast component that executes a temporal-difference learning algorithm, and (b) a slow component, based on an online deterministic annealing algorithm, that adaptively partitions the state-action space according to a dissimilarity measure that belongs to the family of Bregman divergences. The proposed online deterministic annealing algorithm constitutes a competitive-learning neural network that shows robustness with respect to the initial conditions, requires minimal hyper-parameter tuning, and provides online control over the performance-complexity trade-off.

Connected and automated vehicles (CAVs) have seen a rapid surge in interest over the past few years. A lot of focus is being placed on improving the efficiency and robustness of transportation systems by leveraging the sensors and capabilities of CAVs. However, the integration of CAVs into existing traffic infrastructure would give rise to certain issues that must be addressed before the CAVs can be seen ubiquitously on public roads. Since the highway networks are considered permanent investments that are expensive to build and maintain, the priority is to improve the efficiency of the current traffic system. This chapter explores the integration of two of the most common traffic management strategies, namely, ramp metering (RM) and route guidance (RG), into existing highway networks with human-driven vehicles (HDVs). The introduction of CAVs to public roads will engender issues pertaining to safe interactions between CAVs and HDVs. The later part of the chapter addresses the specific problems of improving highway on-ramp merging efficiency by optimally coordinating CAVs. The chapter concludes by presenting a scenario that requires an explicit consideration of interactions between HDVs and CAVs.

The advent and proliferation of the use of Unmanned Aerial Vehicles (UAV’s) poses new challenges to swarm robotics and air defense systems. In this work we consider the problem of defending against adversarial attacks from UAV swarms performing complex maneuvers, driven by multiple, dynamically changing, leaders. We rely on short-time observations of the trajectories of the UAVs and develop a leader detection scheme based on the notion of Granger causality. We proceed with the estimation of the swarm’s coordination laws, modeled by a generalized Cucker-Smale model with non-local repulsive potential functions and dynamically changing leaders, through an appropriately defined iterative optimization algorithm. Similar problems exist in communication and computer networks, as well as social networks over the Internet. Thus, the methodology and algorithms proposed can be applied to many types of network swarms including detection of influential malevolent “sources” of attacks and “miss-information”. The proposed algorithms are robust to missing data and noise. Introduction and Problem Definition While modern high-precision targeting anti-air defenses are capable of taking down a single UAV, when it comes to a large swarm of UAV’s attacking simultaneously, these defences can be rendered ineffective. These problems are even more challenging when in UAV swarms, a few units are managed by humans (we refer to them as “leaders”, while most units “follow”, leading to very effective management of large swarms. When the role of leaders can be dynamically re-assigned the monitoring and defense against such swarms becomes even more difficult. The first question that needs to be addressed in creating a defense against a hostile UAV swarm is understanding the control (coordination) and communication laws governing how the drones move and interact with each other. We view the interconnected problems of modeling and learning the interaction laws of a swarm as one problem that can be analyzed in the microscopic scale as a port-Hamiltonian networked system. We extend existing simulation models, such as the Boids and the Cucker-Smale models, to incorporate interaction, communication and dynamics terms that can capture realistic complex swarm maneuvers and develop corresponding simulation models in the macroscopic domain [8, 10]. Consider an interacting system of N particles and the leader sets L(i), 1 ≤ i ≤ N of cardinality |L(i)| = 1 assigned to each particle representing the index of the leader particle that it is following. We introduce I. a scalable simulation algorithm, based on the Boids model [11], that can capture interaction laws and communication protocols of complex swarm maneuvers, including (a) velocity alignment, (b) spatial cohesion, (c) collision avoidance, and (d) response to dynamically changing leaders: { ẋi = vi v̇i =−c∇Uc(x)−a∇Ua(x,v)+ s∇Us(x) (1) II. a large-scale learning algorithm, based on the generalized CuckerSmale model [1] and automatic differentiation, designed to work on state-of-the-art deep learning platforms that can identify the interaction laws (a)-(d) by observing particle trajectories of position and Extended abstract submitted to the Maryland Robotics Center (MRC) Research Symposium 2021. Research partially supported by the Defense Advanced Research Projects Agency (DARPA) under Agreement No. HR00111990027, by ONR grant N0001417-1-2622, and by a grant from Northrop Grumman Corporation. Authors’ e-mails:{mavridis, nileshs, baras}@umd.edu. Figure 1: Reconstructing complex swarm dynamics. The agents’ trajectories are observed and used to detect leaders and identify a port-Hamiltonian networked system modeling their interaction rules [8]. velocity: { ẋi = vi v̇i = K N ∑ N j=1 ψi j(x(t),v(t)) (2)

In this work we consider the problem of defending against adversarial attacks from UAV swarms performing complex maneuvers, driven by multiple, dynamically changing, leaders. We rely on short-time observations of the trajectories of the UAVs and develop a leader detection scheme based on the notion of Granger causality. We proceed with the estimation of the swarm's coordination laws, modeled by a generalized Cucker-Smale model with non-local repulsive potential functions and dynamically changing leaders, through an appropriately defined iterative optimization algorithm. Similar problems exist in communication and computer networks, as well as social networks over the Internet. Thus, the methodology and algorithms proposed can be applied to many types of network swarms including detection of influential malevolent "sources" of attacks and "miss-information". The proposed algorithms are robust to missing data and noise. We validate our methodology using simulation data of complex swarm movements.

Pose tracking for outdoor rovers is generally a complex task which is further complicated in conditions where a Global Positioning System (GPS) signal is denied such as in planetary exploration, underground mines and covered areas. In these conditions the rover's pose needs to be calculated purely based on the rover's current environment observations. However, conventional wheel odometry is not reliable on rough terrain where wheels are prone to slip and the wheels do not have a common plane of motion due to suspension systems. This paper proposes a Fast State Variable Extension (Fast-SVE) method in which 2D state variables (x, y, yaw) are extended to the full 3D state (x, y, z, roll, pitch, yaw) to achieve effective real time 3D pose tracking of the rover. A particle filter implementation incorporating the Fast-SVE method is used to track the 3D pose of the rover with roll and pitch values used for weighting. An Enhanced Motion Model (EMM) is also proposed to further improve the accuracy of 2D pose prediction in the particle filter.