Many potential applications of artificial intelligence involve making real-time decisions in physical systems while interacting with humans. Automobile racing represents an extreme example of these conditions; drivers must execute complex tactical manoeuvres to pass or block opponents while operating their vehicles at their traction limits1. Racing simulations, such as the PlayStation game Gran Turismo, faithfully reproduce the non-linear control challenges of real race cars while also encapsulating the complex multi-agent interactions. Here we describe how we trained agents for Gran Turismo that can compete with the world's best e-sports drivers. We combine state-of-the-art, model-free, deep reinforcement learning algorithms with mixed-scenario training to learn an integrated control policy that combines exceptional speed with impressive tactics. In addition, we construct a reward function that enables the agent to be competitive while adhering to racing's important, but under-specified, sportsmanship rules. We demonstrate the capabilities of our agent, Gran Turismo Sophy, by winning a head-to-head competition against four of the world's best Gran Turismo drivers. By describing how we trained championship-level racers, we demonstrate the possibilities and challenges of using these techniques to control complex dynamical systems in domains where agents must respect imprecisely defined human norms.

Autonomous car racing is a major challenge in robotics. It raises fundamental problems for classical approaches such as planning minimum-time trajectories under uncertain dynamics and controlling the car at the limits of its handling. Besides, the requirement of minimizing the lap time, which is a sparse objective, and the difficulty of collecting training data from human experts have also hindered researchers from directly applying learning-based approaches to solve the problem. In the present work, we propose a learning-based system for autonomous car racing by leveraging a high-fidelity physical car simulation, a course-progress proxy reward, and deep reinforcement learning. We deploy our system in Gran Turismo Sport, a world-leading car simulator known for its realistic physics simulation of different race cars and tracks, which is even used to recruit human race car drivers. Our trained policy achieves autonomous racing performance that goes beyond what had been achieved so far by the built-in AI, and, at the same time, outperforms the fastest driver in a dataset of over 50,000 human players.

The authors have requested that this preprint be removed from Research Square.

Professional race-car drivers can execute extreme overtaking maneuvers. However, existing algorithms for autonomous overtaking either rely on simplified assumptions about the vehicle dynamics or try to solve expensive trajectory-optimization problems online. When the vehicle approaches its physical limits, existing model-based controllers struggle to handle highly nonlinear dynamics, and cannot leverage the large volume of data generated by simulation or real-world driving. To circumvent these limitations, we propose a new learning-based method to tackle the autonomous overtaking problem. We evaluate our approach in the popular car racing game Gran Turismo Sport, which is known for its detailed modeling of various cars and tracks. By leveraging curriculum learning, our approach leads to faster convergence as well as increased performance compared to vanilla reinforcement learning. As a result, the trained controller outperforms the built-in model-based game AI and achieves comparable overtaking performance with an experienced human driver.

Die umfangreichen Moglichkeiten der modernen IT effizient auszunutzen, ist eine der grosten Herausforderungen fur produzierende kleine und mittlere Unternehmen (KMU). Die tatsachliche Effizienz und der Nutzen eingesetzter IT-Systeme und -Anwendungen sind haufig vollig unklar. Die Grunde dafur sind vielfaltig, jedoch liegt der wohl wichtigste in der fehlenden Bewertungsmoglichkeit der IT-Effizienz, insbesondere des IT-Nutzens. Existierende Modelle fokussieren typischerweise auf die IT-Kosten, die oft leichter zu beurteilen sind als der IT-Nutzen. Trotz der hohen Komplexitat dieser Bewertung darf das Wissen um die Nutzenseite der IT-Effizienz nicht vernachlassigt werden, denn eine effiziente IT-Unterstutzung kann einen Wettbewerbsvorteil fur KMU bedeuten. Die Transparenz uber die Effizienz der eingesetzten IT stellt somit einen kritischen Erfolgsfaktor fur die betrachtete Zielgruppe dar. Da produzierende Unternehmen ihre wertschopfenden Tatigkeiten innerhalb des Auftragsabwicklungsprozesses durchfuhren, ist ein hoher Nutzen der dort eingesetzten IT-Anwendungen besonders wichtig.Um den Mangel an Bewertungsmodellen zu beheben, wird in der vorliegenden Arbeit ein Modell vorgestellt, mit dem die Effizienz der IT-Unterstutzung im Auftragsabwicklungsprozess produzierender KMU systematisch bewertet werden kann. Kern des neuartigen Modells ist ein mehrdimensionales Kennzahlensystem mit besonderer Betonung des durch die IT erbrachten Nutzens. Es bringt die unterschiedlichen Effizienzkategorien in ein ausgewogenes Verhaltnis. Damit lehnt sich das Modell an den Grundgedanken der Balanced Scorecard und das Performance Measurement an. Die Besonderheit des neuen Modells ist vor allem seine Zielgruppenorientierung - sowohl fachlich-inhaltlich als auch was die standardisierte Vorgehensweise betrifft. Eine Reihe von ersten erfolgreichen Projektergebnissen nach Anwendung des Modells zeigt bereits die Funktionsfahigkeit und Notwendigkeit des in der vorliegenden Arbeit beschriebenen Ansatzes. Efficiently utilizing the opportunities opened up by modern IT is one of the most significant challenges for manufacturing small and medium-sized enterprises (SMEs). However, too often it is not clear how efficiently the installed IT systems are being used and how well they actually perform. The reasons are numerous, but probably most important is the lack of methodology and tools for systematically evaluating IT efficiency, especially IT benefit. Existing approaches usually focus on the evaluation of IT costs, which is easier to measure than IT benefit. In spite of the high complexity of the IT benefit, this part of the IT efficiency must not be neglected because an efficient IT support can be a competitive advantage for SMEs. The transparency of the installed IT’s efficiency is thus a critical success factor for the considered target group. A high benefit of IT applications used in the order fulfillment process is especially important because the value-adding activities of manufacturing companies are within this process. Due to the described lack of approaches, this thesis proposes a methodology for systematically evaluating the efficiency of the IT support in the order fulfillment process of manufacturing SMEs. The core of the innovative methodology is a multi-dimensional performance measurement system with emphasis on the benefit rendered by IT, balancing different categories of IT efficiency. Hence, the methodology follows the basic principles of the Balanced Scorecard. The new approach is target-group-specific, both in a functional and in a formal way. A number of first successful results after application of the methodology show the functionality and necessity of the developed approach.