As artificial intelligence (AI) systems are more widely deployed and utilized, they have greater potential to inflict harm or provide benefits to individuals. Designers must consider these concepts when building AI systems, but it is difficult to capture these notions in a mathematical framework. An ethical framework put forth by London and Heidari (Minds Mach, 2024. https://doi.org/10.1007/s11023-024-09696-8 ) provides structure and formal definitions of benefits and harms in relation to an individual’s life plans. This leaves an open question of how these concepts impact the decision-making of an AI system. We leverage their work to provide a direct translation of these theoretical ethical concepts to a standard multi-objective reinforcement learning (MORL) decision model. Using this model, we show that multi-objective rewards are necessary to capture benefits and harms in accordance with their definitions. We demonstrate how to capture these concepts in a MORL system and provide scenarios to highlight the utility of this work.

Autonomous agents are increasingly deployed in complex real-world environments where they must balance multiple goals and adapt to novel scenarios. Unfortunately, learning on the fly to handle new situations can produce dangerous or unstable policies. Creating a diverse set of pre-trained policies overcomes this problem, and evolving those policies using multiple objectives increases their applicability in many domains. Multi-Objective Map Elites (MOME) addresses this issue by searching for a population of policies that are both high performing on a set of objectives as well as diverse across a set of defined behavior metrics, referred to as the Behavior Descriptor (BD). However, the characteristics of the BD that enhance learning and improve search across the behavior and multiobjective spaces in an Reinforcement Learning (RL) setting are still relatively unknown. This work investigates qualities of the BD for MOME that improve search for good policies, in turn, enabling better decision-making. We show that redefining the behavior space can improve coverage of the multi-objective space by up to 36%.

Many critical robot environments, such as healthcare and security, require robots to account for contextdependent criteria when performing their functions (e.g., navigation). Such domains require decisions that balance multiple factors, making it difficult for robots to make contextually appropriate decisions. Multi-Objective Optimization (MOO) methods offer a potential solution by trading off between objectives; however concepts like Pareto fronts are not only expensive to compute but struggle with differentiating among solutions on the Pareto front. This work introduces the Contextual Multi-Objective Path Planning (CMOPP) algorithm, which enables the robot to trade off different complex costs dependent on context. The key insight of this work is to separate the path planning and path cost estimation into two independent steps, thus significantly reducing computation cost without impacting the quality of the resulting path. As a result, CMOPP is able to accurately model path costs, which provide meaningful trade-offs when choosing a path that best fits the context. We show the benefits of CMOPP on case studies that demonstrate its contextual path planning capabilities. CMOPP finds contextually appropriate paths by first reducing the search space up to 99.9% to a near-optimal set of paths. This reduction enables the generation of accurate path cost models, using up to 90% less computation than similar methods.

Evolutionary multiagent systems have been successfully applied to many real world problems, including search and rescue and ocean exploration. However, as the number of agents increases in such problems, the evaluation function captures an individual agent's fitness less and less accurately. As a consequence, agents adopt a small set of acceptable behaviors that are neither optimal nor robust to environmental changes or teammate failures. Fitness shaping, intrinsic fitnesses, or multi-fitness learning alleviate some of these concerns but generally require domain knowledge or the functional form of the evaluation function. In this paper, we introduce Entropy-Based Local Fitnesses (EBLFs) that generate diverse behaviors for agents and produce robust team behaviors without requiring environmental knowledge. The key contribution of EBLFs is to inject a dense, entropy-based fitness into the agents' evolution without interfering with the sparse, high-level system evaluation function. Our results show that the agents using EBLFs learn new skills in difficult environments with sparse feedback without requiring domain knowledge. In addition, EBLFs generated new team-level behaviors that were not defined by a human operator, but beneficial to robust team performance.