Military first responders and civilian experts in international peace missions must make rapid, safety-critical decisions while managing conflict escalation under acute stress. Live field exercises such as the “Native Challenge” provide high realism but are costly and difficult to repeat systematically. The SmartSkills project addresses this gap by implementing Mixed Reality (MR) training with high-fidelity digital twins, standardized crisis scenarios, and a decision support concept that links human factors monitoring to instructor dashboards and debriefing.This work will demonstrate an extended SmartSkills analytics pipeline that focuses on physiological stress and emotion regulation during the Illegal Checkpoint MR scenario. Building on the SmartSkills concept of stress probes—standardized, time-locked stimuli placed at unavoidable key scenes to enable comparable measurements—we will analyze probe-evoked responses for stressors that are (a) artificially applied (e.g., scripted audiovisual cues, time pressure, unexpected instructions) and (b) executed by human agents (role players performing armed threat, separation, searching, and escalating communication).Participants of the pilot study were instrumented with unobtrusive sensing using smart biosignal shirts to capture complementary stress channels: (i) cardiovascular measures: ECG/PPG to estimate heart rate (HR) and heart rate variability (HRV), (ii) respiratory measures for breathing rate, and (iii) eye tracking–based pupillometry (MR headset embedding). Signals were synchronized with scenario events to quantify baseline-corrected, probe-evoked dynamics (peak reactivity, recovery slope, and habituation across repetitions). We will relate these markers to behavioral outcomes relevant to checkpoint management (e.g., de-escalation compliance, timing of key actions, and team coordination).Analytically, we will (1) compare stress signatures between artificial vs. human-enacted probes, (2) estimate individual differences in stress reactivity and emotion regulation (e.g., faster pupillary recovery and HRV rebound as putative indicators of effective regulation), and (3) provide the conceptual basis for developing interpretable multimodal models that map features to categorical stress/risk levels suitable for real-time visualization and after-action review within the SmartSkills decision support framework.At the conference presentation, we will demonstrate representative datasets and results, including event-related physiological traces aligned to stress probes, to illustrate how probe-based MR designs can support repeatable, data-driven evaluation and adaptive training for crisis operations.

Military first responders and also civilian experts in international peace missions must act quickly and unerringly in complex and dangerous situations under stress. Basic skills must be trained in advance as realistically as possible in order to be able to use them efficiently in later, real-life deployment situations and to reduce the risk potential when deployed to crisis areas. Simulations of concrete conflict scenarios are applied in the biannual “Native Challenge” workshop within a military camp area in the Austrian Alpes that is based on the idea of a cooperation between the “UNESCO Chair for Peace Studies” and the Military Command Tyrol. The representation of these simulations where participants experience in exercise scenarios the surprises, conflicts and dangers they can be confronted within a real mission is very costly which prohibits repeated, personalised and focused skill training.The Austrian project SmartSkills aims at providing standardised scenarios from the most diverse areas (behaviour at the checkpoint, negotiation, accident in the minefield, care of the wounded, etc.) interactively, tailored to the participants in terms of stress load, leadership ability and communication behaviour. A Mixed Reality training system allows unlimited repetitions, especially with personalised difficulty adaptations of the scenarios, and provides corresponding content for in-depth debriefings. SmartSkills will offer a highly innovative automated digital analysis of the human factors of the assignment, in particular the decisive situational awareness in critical situations, and uses biosensors to point out cognitive-emotional problem areas that require special attention in skill development. The associated Decision Support System will translate scientifically validated data into pragmatic risk estimates for the attention of the training management.This presentation will for the firstly describe the conceptual outline of the “Native Challenge” which integrates several levels and feedback loops of operational and strategical challenges. Secondly, the setting and the results of the initial requirement study detailed including information about the selected scenarios and use cases, with roles, objects of interest and training objectives. Furthermore, we will describe the technical specification for the Mixed reality system, its general system architecture, with the outline of wearable bio-signal sensors for psychophysiological monitoring, the graphical processing including realistic object visualisation and digital twins that were scanned from the real operational sites. We will provide details about the study plan, its research hypotheses, in particular, in the context of the evaluation of the training and psychophysiological key performance indicators. Finally, we will describe the anchoring in the international context and the acceleration of developments through exchange with European initiatives.SmartSkills is researching a new dimension of particularly realistic visualisations through the use of innovative digital twins: with highly accurate measurement technology and AI-supported evaluation software, internationally relevant environments can be experienced directly in the simulation centre, thereby increasing the realistic immersion.

Autonomes Fahren ist längst keine Science-Fiction mehr, sondern eine Realität, die sich schnell entwickelt. In diesem Artikel werfen wir einen Blick auf Level 4-Autonomie, die eine bedeutende Stufe auf dem Weg zu vollständig autonomen Fahrzeugen darstellt. Wir werden untersuchen, was für die Erreichung dieser Stufe notwendig ist, warum ein Digitaler Zwilling aktuell unerlässlich ist und wie er in die Entwicklung und Implementierung integriert wird. Dabei werden wir auch die aktuellen Herausforderungen thematisieren und einen Ausblick auf die Zukunft des Autonomen Fahrens bieten.

AbstractWithin the EU Horizon 2020 project SHOW (GA No 875530) auxiliary measures, via evaluating and adapting physical road infrastructure (for instance lane markings, traffic signs, sight distances), as well as via digital support were explored for their potential contribution to enabling automated shared mobility services on the road environment. This line of research was then followed up on in the EU Horizon Europe Project AUGMENTED CCAM (GA 101,069,717), tackling more specifically with PDI support for automated mobility. This chapter presents the activities and findings of these two projects in relation to physical infrastructure adaptations for automated vehicles for two pilot sites in Austria.

Autonomous driving is no longer science fiction but a rapidly developing reality. In this article, we will take a look at Level 4 autonomy, a significant milestone on the path to fully autonomous vehicles. We will examine what is necessary to achieve this level, why a digital twin is currently indispensable, and how it is integrated into development and implementation. We will also discuss the current challenges and provide an outlook on the future of autonomous driving.

Pre-Sale validation, certification, homologation as well as after-sale monitoring, continuous (re-)homologation during operation of automated vehicles of all levels of automation are still posing major challenges to OEMS, suppliers, developers and certification bodies. These challenges are also reflected in current legislation and included in the latest assessment documents proposed by the UNECE (NATM). In this paper, we present building bricks for addressing these challenges. The building bricks originate from the current and final results of two European sister EUREKA projects. To handle complexity, specific driving functions (composed of the perceive-interpret-act cycle) for relevant operational domains and different perception pipelines were used as representative examples in the projects. As it is well known that the number of kilometres to be driven to thoroughly test the automated driving functionalities by far extends the feasible number. Therefore, one major output is how to generate realistic reference data for simulation via detailed measurements and how these measurements can enhance the various stages in the validation process. This data includes not only general environment (static objects) and scenario measurements (dynamic objects), but also details information on material properties, e.g. in terms of reflectivity, and the real-time monitoring of additional environmental conditions. For selected driving functions and for the environment perception systems the two EUREKA sister projects developed various technologies, methodologies and tools as potential building bricks for the NATM multi-pillar framework; where a “best-of” selection presented within this paper.

This conference paper presents the mid-term results of the two EUREKA projects “testEPS” and “Central System”. TestEPS addresses the certification of autonomous driving systems, while Central System focuses on infrastructure solutions for a connected vehicle system. Both integrate the same four technological fields: simulation and virtual testing, realworld testing, HD mapping, and communication, to realize their vision and to achieve their individual goals. The paper presents application use cases for each field as utilized in the respective project. Furthermore, the importance of collaboration between these two projects and respective technologies in advancing the state-of-the-art in autonomous and connected vehicle technologies is highlighted.

In recent years, verification and validation processes of automated driving systems have been increasingly moved to virtual simulation, as this allows for rapid prototyping and the use of a multitude of testing scenarios compared to on-road testing. However, in order to support future approval procedures for automated driving functions with virtual simulations, the models used for this purpose must be sufficiently accurate to be able to test the driving functions implemented in the complete vehicle model. In recent years, the modelling of environment sensor technology has gained particular interest, since it can be used to validate the object detection and fusion algorithms in Model-in-the-Loop testing. In this paper, a practical process is developed to enable a systematic evaluation for perception–sensor models on a low-level data basis. The validation framework includes, first, the execution of test drive runs on a closed highway; secondly, the re-simulation of these test drives in a precise digital twin; and thirdly, the comparison of measured and simulated perception sensor output with statistical metrics. To demonstrate the practical feasibility, a commercial radar-sensor model (the ray-tracing based RSI radar model from IPG) was validated using a real radar sensor (ARS-308 radar sensor from Continental). The simulation was set up in the simulation environment IPG CarMaker® 8.1.1, and the evaluation was then performed using the software package Mathworks MATLAB®. Real and virtual sensor output data on a low-level data basis were used, which thus enables the benchmark. We developed metrics for the evaluation, and these were quantified using statistical analysis.

Validation and certification enabling the operation of automated vehicles may perhaps be the biggest challenge for the developers of such systems. While there is a consensus about the outstanding problem and the related questions, there is, unfortunately, no widely accepted and feasible solution to this problem. The classical automotive testing and validation approaches for certifying automated vehicles would require several millions of km of on-road testing time and are unfeasible. Simulation techniques can potentially be utilized for this purpose, but the fidelity and modelling accuracy requirements are unknown. This issue is addressed by the two recent sister projects, ‘testEPS’ and ‘Central System’, co-funded via the EUREKA network and the Austrian and Hungarian Ministry of Transportation. The current paper aims to introduce both projects together with their respective goals, implementation plans and expected outcomes. They originate from an exploratory phase with initial cross-project cooperation. Due to the joint genesis and unique structure cross-projects synergies effects are expected to be realised to support future verification and validation of ADAS/AD systems, including standardization.

A spectacular measurement campaign was carried out on a real-world motorway stretch of Hungary with the participation of international industrial and academic partners. The measurement resulted in vehicle based and infrastructure based sensor data that will be extremely useful for future automotive R&D activities due to the available ground truth for static and dynamic content. The aim of the measurement campaign was twofold. On the one hand, road geometry was mapped with high precision in order to build Ultra High Definition (UHD) map of the test road. On the other hand, the vehicles—equipped with differential Global Navigation Satellite Systems (GNSS) for ground truth localization—carried out special test scenarios while collecting detailed data using different sensors. All of the test runs were recorded by both vehicles and infrastructure. The paper also showcases application examples to demonstrate the viability of the collected data having access to the ground truth labeling. This data set may support a large variety of solutions, for the test and validation of different kinds of approaches and techniques. As a complementary task, the available 5G network was monitored and tested under different radio conditions to investigate the latency results for different measurement scenarios. A part of the measured data has been shared openly, such that interested automotive and academic parties may use it for their own purposes.

This paper provides an overview of a novel system architecture to support first responders in coping with dynamically changing situations, as well as help them to allocate human resources more efficiently. The system supports the collection and fusion of multi-sensor data for the generation of a near real-time situational picture. We employ a context-sensitive augmented reality (AR) assistance and feedback system and a role-based data distribution module, to enable more efficient interaction between command center and mobile teams and to reduce the risk of information overload. Our system integrates TETRA, since it is a widely used radio standard for first responders, and complements it with other broadband systems like LTE and WiFi to enable broadband multimedia services.

Smartphones, as highly portable networked computing devices with embedded sensors including GPS receivers, are ideal platforms to support context-aware language learning. They can enable learning when the user is en-gaged in everyday activities while out and about, complementing formal language classes. A significant challenge, however, has been the practical implementation of services that can accurately identify and make use of context, particularly location, to offer meaningful language learning recommendations to users. In this paper we review a range of approaches to identifying context to support mobile language learning. We consider how dynamically changing aspects of context may influence the quality of recommendations presented to a user. We introduce the MASELTOV project’s use of context awareness combined with a rules-based recommendation engine to present suitable learning content to recent immigrants in urban areas; a group that may benefit from contextual support and can use the city as a learning environment.

Human Interaction with mobile devices has recently been estab-lished as application field in eye tracking research. Current technologies for gaze recovery on mobile displays cannot enable fully natural interaction with the mobile device: users are condi-tioned to interact with tightly mounted displays or distracted by markers in their view. We propose a novel approach that cap-tures point-of-regards (PORs) with eye tracking glasses (ETG) and then uses computer vision methodology for the robust local-ization of the smartphone in the head camera video. We present an integrated software package, i.e., the Smartphone Eye Track-ing Toolbox (SMET) that enables accurate gaze recovery on mobile displays with heat mapping of recent attention. We re-port the performance of the computer vision approach and demonstrate it with various natural interaction scenarios using the SMET Toolbox, enable ROI settings on the mobile display and show results from eye movement analysis, such as, ROI dwell time and statistics on eye gaze event (saccades, fixations).

Immigration imposes a range of challenges with the risk of social exclusion. As part of a comprehensive suite of services for immigrants, the MASELTOV game seeks to provide both practical tools and innovative learning services via mobile devices, providing a readily usable resource for recent immigrants. We introduce advanced results, such as the game-based learning aspect in the frame of recommender services, and present the rationale behind its interaction design. Benefits and implications of mobile platforms and emergent data capture techniques for game-based learning are discussed, as are methods for putting engaging gameplay at the forefront of the experience whilst relying on rich data capture and analysis to provide effective learning solutions.

While mobile applications typically offer access to standardized ‘basic’ geo-content, there is evidence in the human sciences that people ac-tually prefer subjective information sources for decision making, e.g. personal stories about experiences by family and friends. The success story of content communities in web applications confirms the wide acceptance of innovative information systems that offer the potential to consume, to produce and to rate personalized data. Therefore in this paper an approach for a mobile, interactive and integrated system is presented that do not only deliver basic geo-referenced information, but also allow the users to create location aware information for themselves and other users: information like hints and personal experience will be geo-referenced, time stamped and annotated with text and keyword information. It is stored and exchanged between community members. This individual information creates information content which is continuously growing and updated. Image based and text based information retrieval in combination with location information are used to provide easy access to relevant information. In this paper we outline the system architecture and components that enable these new approaches also providing augmented reality navigation. The approach was tested in a field test study and results and open issues are given. The system worked well and could be applied to future experiments in order to gain more insight in the mobile users’ behavior in real contexts.

KurzdarstellungAuf Grund ständig steigender Besucherzahlen in Großschutzgebieten ist bei der Angebotsplanung für Besucher eine langfristige Sicherung kultureller und natürlicher Gegebenheiten wichtig. Im Rahmen des Projekts BALANCE wurden dazu eine GPS-basierte mobile Applikation zur Informationsvermittlung und zur Datenaufzeichnung, ein Datenmanagementtool zur Angebotserstellung und ein Tool zur Besucheranalyse entwickelt. Die gesammelten Daten wurden mit Hilfe des entwickelten Analysetools ausgewertet. Für das Schutzgebietsmanagement wurden dadurch wichtige Informationen über das Verhalten der Besucher (z.B. bevorzugte Routen, Anzahl und Dauer von Stopps, etc.) ersichtlich. Die Entwicklung erfolgte in Zusammenarbeit mit zwei Nationalparks und wurde abschließend im Rahmen eines Pilottests durch Besucher und Nationalparkbetreiber getestet und evaluiert.

(Lucas Paletta, JOANNEUM RESEARCH ForschungsgesmbH, 8010 Graz, Austria, lucas.paletta@joanneum.at) (Patrick Luley, JOANNEUM RESEARCH ForschungsgesmbH, 8010 Graz, Austria) (Katrin Amlacher, JOANNEUM RESEARCH ForschungsgesmbH, 8010 Graz, Austria) (Alexander Almer, JOANNEUM RESEARCH ForschungsgesmbH, 8010 Graz, Austria) (Reinhard Sefelin, CURE Center for Usability Research and Engineering, 1110 Vienna, Austria, sefelin@cure.at) (Manfred Tscheligi, CURE Center for Usability Research and Engineering, 1110 Vienna, Austria) (Joachim Ortner, C.C.Com Andersen & Moser GmbH, 8074 Grambach, Austria, jortner@cccom.at) (Burkhard Moser, C.C.Com Andersen & Moser GmbH, 8074 Grambach, Austria) (Jutta Manninger Graz AG – Verkehrsbetriebe, 8010 Graz, Austria, j.manninger@grazag.at) (Walter Scheitz, GEFAS Steiermark, 8010 Graz, Austria, gefas@generationen.at) (Regina Wallner, GEFAS Steiermark, 8010 Graz, Austria) (Marianne Hammani-Birnstingl, Verein DANAIDA, 8020 Graz, Austria, marianne.hammani-birnstingl@danaida.at) (Verena Radoczky, Mentz Datenverarbeitung Austria GmbH, 1060 Wien, Austria, radoczky@mentzdv.at) (Otto Rath, ISOP Graz, 8020 Graz, Austria, otto.rath@isop.at)

Mobile vision services have recently been proposed for the support of urban nomadic users. While camera phones with image based recognition of urban objects provide intuitive interfaces for the exploration of urban space and mobile work, similar methodology can be applied to vision in mobile robots and autonomous aerial vehicles. A major issue for the performance of the service - involving indexing into a huge amount of reference images - is ambiguity in the visual information. We propose to exploit geo-information in association with visual features to restrict the search within a local context. In a mobile image retrieval task of urban object recognition, we determine object hypotheses from (i) mobile image based appearance and (ii) GPS based positioning, and investigate the performance of Bayesian information fusion with respect to benchmark geo-referenced image databases (TSG-20, TSG-40). This work specifically proposes to introduce position information as geo-contextual priors for geo-attention based object recognition to better prime the vision task. The results from geo-referenced image capture in an urban scenario prove a significant increase in recognition accuracy (> 10%) when using the geo-contextual information in contrast to omitting geo-information, the application of geo-attention is capable to improve accuracy by further > 5%.