The ability of a robot to improve its performance on a task can be critical, especially in poorly known and non-stationary environments where the best action or strategy is dependent upon the current state of the environment. In such systems, a good estimate of the current state of the environment is key to establishing high performance, however quantified. In this paper, we present an approach to state estimation in poorly known and non-stationary mobile robot environments, focusing on its application to a mine collection scenario, where performance is quantified using reward maximization. The approach is based on the use of augmented Markov models (AMMs), a sub-class of semi-Markov processes. We have developed an algorithm for incrementally constructing arbitrary-order AMMs on-line. It is used to capture the interaction dynamics between a robot and its environment in terms of behavior sequences executed during the performance of a task. For the purposes of reward maximization in a non-stationary environment, multiple AMMs monitor events at different timescales and provide statistics used to select the AMM likely to have a good estimate of the environmental state. AMMs with redundant or outdated information are discarded, while attempting to maintain sufficient data to reduce conformation to noise. This approach has been successfully implemented on a mobile robot performing a mine collection task. In the context of this task, we first present experimental results validating our reward maximization performance criterion. We then incorporate our algorithm for state estimation using multiple AMMs, allowing the robot to select appropriate actions based on the estimated state of the environment. The approach is tested first with a physical robot, in a non-stationary environment with an abrupt change, then with a simulation, in a gradually shifting environment.

Multirobot coordination remains a challenging problem with potentially high impact on space applications if solved. The challenge is to enable robots to work together in an intelligent manner to execute a global task. This paper describes a market- based approach suitable for complex multirobot space applications. The market approach has had considerable success in the multirobot coordination domain. Previously published work introduced the market architecture and showed that it is inherently distributed, but can also opportunistically form centralized sub-groups to improve efficiency, and thus approach optimality. This paper focuses on the space application domain, and more specifically, presents simulation results for market-based coordination of a group of heterogeneous robots engaged in information gathering on a Martian outpost.

This paper presents a market-based, multi-robot planning capability, designed as part of a distributed, layered architecture for multi-robot control and coordination. More specifically, we are developing an extension to the traditional three-layered robot architecture that enables robots to interact directly at each layer - at the behavioral level, the robots create distributed control loops; at the executive level, they synchronize task execution; at the planning level, they use market-based techniques to assign tasks and allocate resources. The market-based planning layer of each robot has two main components: (1) a trader that participates in the market, auctioning and bidding on tasks; (2) a scheduler that determines task feasibility and cost for the trader, and interacts with the executive layer for task execution. This paper focuses on the planning level, detailing the architecture, our current implementation, and planned future extensions. We show how the architecture (in particular, the planning layer) has been applied to a Mars exploration scenario involving the characterization of scientifically "interesting" rocks. We also present preliminary simulation results exploring market and scenario parameters.

In an attempt to solve as much of the AAAI Robot Challenge as possible, five research institutions representing academia, industry, and government integrated their research into a single robot named GRACE. This article describes this first-year effort by the GRACE team, including not only the various techniques each participant brought to GRACE but also the difficult integration effort itself.

This paper describes a market-based planning mechanism used for task and resource allocation within a larger distributed, multi-robot control and coordination architecture. We are developing an extension to the traditional three-layered robot architecture that enables robots to interact directly at each layer -- at the behavioral level, the robots create distributed control loops; at the executive level, they synchronize task execution; at the planning level, they use market-based techniques to allocate tasks and resources. This paper focusses on the market-based planning layer, which is comprised of two main components: a trader that participates in the market, auctioning and bidding on tasks; and a scheduler that determines task feasibility and cost for the trader, and interacts with the executive layer for task execution.

This paper presents an architecture that enables multiple robots to explicitly coordinate actions at multiple levels of abstraction. In particular, we are developing an extension to the traditional three-layered robot architecture that enables robots to interact directly at each layer — at the behavioral level, the robots create distributed control loops; at the executive level, they synchronize task execution; at the planning level, they use market-based techniques to assign tasks, form teams, and allocate resources. We illustrate these ideas through applications in multi-robot assembly, multi-robot deployment, and multi-robot mapping.

Improving the performance of agent-based systems is a challenging problem requiring both system evaluation and appropriate modification of the agent's policy or controller. This dissertation presents work in this problem domain, focusing on the development of an on-line, real-time method for modeling the interaction dynamics between a situated agent and its environment. The methods and experimentation presented in this thesis aim to show that the evaluation of agent-environment interaction dynamics can be effective and efficient in improving the performance of agents in challenging problem domains. The dissertation is first motivated both by an examination of how behavior-based control provides a rich substrate for the evaluation of interaction dynamics, and by the unifying experimental theme (mobile robot foraging). The majority of the dissertation focuses on on-line learning of augmented Markov models (AMMs), a novel version of semi-Markov processes. The approach utilizes AMMs to capture agent-environment interaction dynamics in terms of the history of behaviors executed while performing a task. These models provide the data that are used on-line and in real-time to evaluate the system and suggest task-dependent, performance-improving modifications to the agent's behavior. An AMM construction algorithm is presented that allows incremental generation with little computational overhead, making it feasible for on-line, real-time applications. The algorithm is able to represent non-first-order Markovian systems in first-order form by dynamically adjusting models through the use of higher-order statistics. This ability to represent higher-order Markovian characteristics provides the expressiveness to accommodate systems with rich interaction dynamics. The on-line, real-time modeling approach using AMMs in conjunction with behavior-based control is demonstrated as effective in both stationary and non-stationary problem domains. Several challenging robotics applications are examined in the stationary domain (fault detection, affiliation determination, hierarchy restructuring) and the non-stationary domain (regime detection, reward maximization). The AMM-based evaluations used in these applications include statistical hypothesis tests and expectation calculations from Markov chain theory. Experimental results are presented for each of the methods and applications discussed. Finally, some of the statistical distribution issues involving AMMs and their utilization in this work are addressed through an empirical comparison with a non-parametric alternative.

We present an approach to the detection of global environmental regime changes by a mobile robot performing a task. The approach is based on the use of augmented Markov models (AMMs), a variation of semi-Markov process. We have developed an algorithm that constructs AMMs online and in real-time with little overhead. AMMs are a general tool for capturing the interaction dynamics between a robot and its environment using the history of behavior executed by the robot. We extend AMMs to regime detection, using multiple models to monitor events at different time scales and provide statistics to detect regime changes at those time scales. This approach has been successfully implemented using a physical mobile robot performing a land mine collection task. In the context of this task we present experimental results, first validating our approach, then demonstrating a more complex proportion-maintaining scenario of the land mine collection task. Finally, we present results using an alternative reward maximization decision criterion in the same task

In this chapter, we demonstrate the e ectiveness of behavior-based control in facilitating the development and evaluation of multi-robot controllers that are: (1) robust to robot failures, and (2) easily modi ed to facilitate development of the controller variation that su ciently satis es the design requirements for the task. Our experimental focus here is distributed multi-robot collection, a class of tasks that includes de-mining and toxic waste clean-up. We demonstrate a basic, homogeneous multi-robot controller for the collection task, then show how to easily derive two heterogeneous, spatio-temporal variations with markedly di erent performance properties. We evaluate the desirability of these controllers with respect to design requirements involving inter-robot interference, time-to-completion, and energy expenditure. The data for evaluation come from experiments using four physical mobile robots performing the three variations of the collection task.

: We demonstrate the effectiveness of behavior-based control in facilitating the development and evaluation of multi-robot controllers that are: (1) robust to robot failures, and(2) easily modified to facilitate development of the controller variation that sufficiently satisfies the design requirements for the task. Our experimental focus here is distributed multi-robot collection, a class of tasks that includes de-mining and toxic waste clean-up. We demonstrate a basic multi-robot controller for the collection task, then show how to easily derive two spatio-temporal variations with markedly different performance properties. We evaluate the desirability of these controllers with respect to design requirements involving inter-robot interference, time-to-completion, and energy expenditure. The data for evaluation come from experiments using four physical mobile robots performing the three variations of the collection task.

This thesis proposal explores off-line and on-line methods for evaluating the dynamics of mobile robotic systems. Our exploration of off-line ie, pre post-experimental methods centers on the use of interference between robots as one pragmatic tool for evaluating multi-robot controllers. We show how key issues in multi-robot control can be addressed using interference, a directly measurable property of a multi-robot system. We discuss how behavior arbitration schemes, ie, the choice of controllers, can be made and adjusted using interference. To provide a strong formal basis, our use of interference is closely linked to several statistical methods, including hypothesis tests and spatial statistics.

We present an approach to reward maximiza-tion in a non-stationary mobile robot environment. The approach works within the realistic constraints of limited local sensing and limited a priori knowledge of the environment. It is based on the use of augmented Markov models (AMMs), a general modeling tool we have developed. AMMs are essentially Markov chains having additional statistics associated with states and state transitions. We have developed an algorithm that constructs AMMs on-line and in real-time with little computational and space overhead, making it practical to learn multiple models of the interaction dynamics between a robot and its environment during the execution of a task. For the purposes of reward maximiza-tion in a non-stationary environment, these models monitor events at increasing intervals of time and provide statistics used to discard redundant or outdated information while reducing the probability of conforming to noise. We have successfully implemented this approach with a physical mobile robot performing a mine collection task. In the context of this task, we rst present experimental results validating our reward max-imization criterion in a stationary environment. We then incorporate our algorithm for redundant/outdated information reduction using multiple models and apply the approach to a non-stationary environment with an abrupt change. Finally, we apply the technique to a simulated version of the task with a gradually shifting environment.

This technical report presents augmented Markov models (AMMs), and provides detailed descriptions of their structure and one model construction algorithm. Augmented Markov models are essentially probabilistic transition networks similar to hidden Markov models (HMMs), except that the hidden state assumption is removed. Additional statistics (augmentations) are maintained in the links and nodes of AMMs and may be employed in model construction and utilization. The model construction algorithm we present is designed to have relatively low computational and space overheads, and provide useful models on-line and in real-time.

In complex, dynamic and uncertain environments extending from disaster rescue missions, to future battlefields, to monitoring and surveillance tasks, to virtual training environments, to future robotic space missions, intelligent agents will play a key role in information gathering and filtering, as well as in task planning and execution. Although physically distributed on a variety of platforms, these agents will interact with information sources, network facilities, and other agents via cyberspace, in the form of the Internet, Intranet, the secure defense communication network, or other forms of cyberspace. Indeed, it now appears well accepted that cyberspace will be (if it is not already) populated by a vast number of such distributed, individual agents.

We show how various levels of coordinated behavior may be achieved in a group of mobile robots by using a model of the interaction dynamics between a robot and the environment. We present augmented Markov models (AMMs) as a tool for capturing such interaction dynamics on-line an in real-time, with little computational and storage overhead. We briefly describe the structure of AMMs, then demonstrate the application of the model for resolving group coordination issues arising from three sources: individual performance, group affiliation, and group performance. Corresponding respectively to these are the three experimental examples we present - fault detection, group membership based on ability and experience, and dynamic leader selection.

Article Coordinating mobile robot group behavior using a model of interaction dynamics Share on Authors: Dani Goldberg Univ. of Southern California, Los Angeles Univ. of Southern California, Los AngelesView Profile , Maja J. Matarić Univ. of Southern California, Los Angeles Univ. of Southern California, Los AngelesView Profile Authors Info & Claims AGENTS '99: Proceedings of the third annual conference on Autonomous AgentsApril 1999 Pages 100–107https://doi.org/10.1145/301136.301172Online:01 April 1999Publication History 24citation601DownloadsMetricsTotal Citations24Total Downloads601Last 12 Months7Last 6 weeks1 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access