User-friendly modeling and virtual simulation of urban traffic scenarios with different types of interacting agents such as pedestrians, cyclists and autonomous vehicles remains a challenge. We present CARJAN, a novel tool for semi-automated generation and simulation of such scenarios based on the multi-agent engineering framework AJAN and the driving simulator CARLA. CARJAN provides a visual user interface for the modeling, storage and maintenance of traffic scenario layouts, and leverages SPARQL Behavior Tree-based decision-making and interactions for agents in dynamic scenario simulations in CARLA. CARJAN provides a first integrated approach for interactive, intelligent agent-based generation and simulation of virtual traffic scenarios in CARLA.

There are many established semantic Web standards for implementing multi-agent driven applications. The AJAN framework allows to engineer multi-agent systems based on these standards. In particular, agent knowledge is represented in RDF/RDFS and OWL, while agent behavior models are defined with Behavior Trees and SPARQL to access and manipulate this knowledge. However, the appropriate definition of RDF/RDFS and SPARQL-based agent behaviors still remains a major hurdle not only for agent modelers in practice. For example, dealing with URIs is very error-prone regarding typos and dealing with complex SPARQL queries in large-scale environments requires a high learning curve. In this paper, we present an integrated development environment to overcome such hurdles of modeling AJAN agents and at the same time to extend the user community for AJAN by the possibility to leverage Large Language Models for agent engineering.

The need for adaptive automated assembly planning is gaining importance as product variety increases and production times shrink, especially in sectors like automotive manufacturing. Answer Set Programming (ASP) is a proven method for solving combinatorial problems such as assembly planning. Inductive logic programming (ILP) on the other hand enables interpretable logic-based machine learning with minimal training data. And the Asset Administration Shell (AAS) can provide essential planning information for both the product and its production environment. Combining ASP-based planning with ILP-learned expert knowledge promises adaptive automation, which, when integrated with AAS, can seamlessly fit into existing production environments. This work-in-progress paper presents our work on ASP-based assembly planning and outlines the integration of learned worker-based expert knowledge and AAS.

Several applications in Industry 4.0 (worker training, navigation planning, synthetic data generation, etc.) require the simulation of smart workers in a 3D environment or Industrial Metaverse. Digital twin technology has primarily focused on the simulation and integration of technical components, and the resulting frameworks and approaches are insufficient for worker simulation in a geometrical sense. In this work, we present how smart workers can be simulated with the MOSIM Framework, a distributed modular framework for simulating human agent motions and behavior in a digital reality. It can simulate the behavior of smart workers, their interaction with the environment and objects, and the corresponding movements. It is integrated into several 3D engines and can exchange information with other simulation components inside an Industrial Metaverse. The approach was applied for final assembly tasks and is available as an open-source project.

Integrating data sources and connecting participants in heterogeneous environments is a challenging task that requires extensive expert knowledge about the nature of the systems involved. Even when this knowledge is given in the form of manuals or code documentation, or can be derived by a human expert by interpreting the source code, putting this knowledge to use to actually integrate data sources is mostly entirely left to the human and not exploitable by the system itself. Established standards, such as data spaces or the Asset Administration Shell, aim to provide support with connecting participants in heterogeneous environments, but often fall short when it comes to intuitive interoperability. With this work, we propose an approach to augment existing systems with machine usable expert knowledge via so-called semantic support points: a concept for minimal implementations, adding the power of Semantic Web technologies like semantic queries, as well as the possibility to model expert knowledge in ontologies, to existing systems. One core goal of ours is leaving existing standards untouched, like secure communication between data space participants or industry environments described by Asset Administration Shells, and adding semantic information as an optional, volatile layer on top, generated by externally managed expert knowledge in the form of semantic transformation rules.

The development of Semantic Web-enabled intelligent agents and multi-agent systems still remains a challenge due to the fact that there are hardly any agent engineering frameworks available for this purpose. To address this problem, we present AJAN, a modular framework for the engineering of agents that builds on Semantic Web standards and Behavior Tree technology. AJAN provides a web service-based execution and modeling environment in addition to an RDF-based modeling language for deliberative agents where SPARQL-extended behavior trees are used as a scripting language to define their behavior. In addition, AJAN supports the modeling of multi-agent coordination protocols while its architecture, in general, can be extended with other functional modules as plugins, data models, and communication layers as appropriate and not restricted to the Semantic Web.

The efforts that go into setting up stable, efficient, fault-tolerant Multi-Agent Systems are twofold: Modelling and designing agent interactions with both environment and other agents; and transfering the model into an actual executable implementation of the system. For formal modeling, notations in process calculi, such as Milner’s Calculus for Communicating Systems (CCS), have shown to be helpful, as they allow for formal model checking of the system before implementation. Typically, translating from such notations to an executable counterpart comes with pitfalls, as constraints and limitations set by the chosen programming language or software framework make a one-to-one mapping from the formal expression to executable code difficult. In this paper, we for this investigate the similarities between formal expressions in CCS, and executable behavior trees as agent programs. Our study confirms that a semantic preserving mapping between both exist, and we demonstrate how to exploit these similarities to transfer CCS expressions to semantically equivalent, executable agent programs.

The concept of the cloud-to-things continuum addresses advancements made possible by the widespread adoption of cloud, edge, and IoT resources. It opens the possibility to combine classical symbolic AI and advanced machine learning approaches in a meaningful way. In this paper we present a thing registry and an agent-based orchestration framework which we combine to support semantic orchestration of IoT use cases across several federated cloud environments. We illustrate the approach with a use case from an assisted living scenario.

The availability of sensor technology in home and building automation offers new opportunities for AI applications. Machine learning (ML) methods can recognize device patterns and profiles based on sensor data and make energy-relevant predictions. At a higher level, the detected patterns can in turn be used to learn activity patterns. In this paper, we present a hybrid approach to augment ML-based outputs with inductive logic-based learning techniques. We present evaluation results and illustrate an application to agent-supported critical situation detection in the assisted living domain.

In the area of autonomous driving there is a need to flexibly configure and simulate more complex individual pedestrian behavior in critical traffic scenes which goes beyond predefined behavior simulation. This paper presents a novel human-oriented, agent-based pedestrian simulation framework, named HAIL, that addresses this challenge. HAIL allows to simulate human pedestrian behavior through means of imitation learning by virtual agents. For this purpose, HAIL combines the 3D traffic simulation environment OpenDS with an integrated imitation learning environment and hybrid agents with AJAN. For predictive behavior planning on the tactical and strategical level, AJAN is extended with Answer Set Programming. For pedestrian behavior imitation learning on the operational level, HAIL utilizes the module InfoSalGAIL for generation of pedestrian paths learned from demonstration by its human counterpart as expert. Among others, an application example has been demonstrated that HAIL can be applied to solve a common challenge in the Neural Network domain, namely the out-of-distribution (OOD), e.g. never shown scenarios would raise an uncertainty prediction level, by unison work of the two different behavior generation frameworks.

The concept of the cloud-to-thing continuum addresses advancements made possible by the widespread adoption of cloud, edge, and IoT resources. It opens the possibility of combining classical symbolic AI with advanced machine learning approaches in a meaningful way. In this paper, we present a thing registry and an agent-based orchestration framework, which we combine to support semantic orchestration of IoT use cases across several federated cloud environments. We use the concept of virtual sensors based on machine learning (ML) services as abstraction, mediating between the instance level and the semantic level. We present examples of virtual sensors based on ML models for activity recognition and describe an approach to remedy the problem of missing or scarce training data. We illustrate the approach with a use case from an assisted living scenario.

For years, the manufacturing industry has been investing substantial amounts of research and development work for the implementation of hybrid teams of human workers and robotic units. The composition of hybrid teams requires an optimal coordination of individual players with fundamentally different characteristics and skills. In this paper, we present a highly configurable simulation environment supporting end-users, e.g. manufacturing planners, to optimally prepare, evaluate and improve the collaboration of hybrid teams in the scope of production lines. For generating the optimal task assignment, a GPU-based high-performance optimizer is introduced into the simulation environment. The framework is embedded in a web-based distributed infrastructure that models and provides the involved components (digital human models, robots, visualization environment) as resources. We illustrate the approach with a use case originating from the aircraft industry.

The imitation learning of complex pedestrian behavior based on visual input is a challenge due to the underlying large state space and variations. In this paper, we present a novel visual attention-based imitation learning framework, named InfoSalGAIL, for end-to-end imitation learning of (safe, unsafe) pedestrian navigation policies through visual expert demonstrations empowered by eye fixation sequence and augmented reward function. This work shows the relation in latent space between the policy estimated trajectories and visual-attention map. Moreover, the conducted experiments revealed that InfoSalGAIL can significantly outperform the state-of-the-art baseline InfoGAIL. In fact, its visual attention-empowered imitation learning tends to much better generalize the overall policy of pedestrian behavior leveraging apprenticeship learning to generate more human-like pedestrian trajectories in virtual traffic scenes with the open source driving simulator OpenDS. InfoSalGAIL can be utilized in the process of generating and validating critical scenarios for adaptive driving assistance systems.

In the production industry, in recent years more and more hybrid teams of workers and robots are being used to improve flexible processes. The production environment of the future will include hybrid teams in which workers cooperate more tightly together with robots and virtual agents. The virtual validation of such teams will require simulation environments in which various safety and productivity issues can be evaluated. In this paper, we present a framework for 3D simulation of hybrid teams in production scenarios based on an agent framework that can be used to evaluate critical properties of the planned production environment and the dynamic assignment of tasks to team members. The framework is embedded in a web-based distributed infrastructure that models and provides the involved components (digital human models, robots, visualization environment) as resources. We illustrate the approach with a use case in which a human-robot team works together in an aircraft manufacturing scenario.

The manufacturing industry has been putting considerable research and development work into the implementation of hybrid teams consisting of workers, robots and assistance systems for several years. The production environment of the future will consist of hybrid teams of humans closely collaborating with robots and virtual agents. Hybrid teams require adaptive control, coordination of interactions between team members as well as reactive behavior for adapting production processes just in time and on demand e.g. when facing unexpected obstacles. In this paper we present a framework for 3D simulation of hybrid teams in production scenarios based on an agent framework and motion generation which can be used for evaluating critical properties of the planned production setting and dynamical assignment of tasks to team members. We illustrate the approach with a use case in which a human-robot team collaborates in a car manufacturing scenario.

The use of autonomous robots in both industry and every day life has increased significantly in the recent years. A growing number of robots is connected to and operated from networks, including the World Wide Web. Consequently, the Robotics community is exploring and adopting REST (Representational State Transfer) architectural principles and considers the use of Linked Data technologies as fruitful next step. However, we observe a lack of concise and stable specifications of how to properly leverage the RESTful paradigm and Linked Data concepts in the Robotics domain. Introducing the notion of Linked Robotic Things, we provide a minimalistic, yet well-defined specification covering a minimal set of requirements with respect to the use of HTTP and RDF.