Search engines are perceived as a reliable source for general information needs. However, finding the answer to medical questions using search engines can be challenging for an ordinary user. Content can be biased and results may present different opinions. In addition, interpreting medically related content can be difficult for users with no medical background. All of these can lead users to incorrect conclusions regarding health related questions. In this work we address this problem from two perspectives. First, to gain insight on users' ability to correctly answer medical questions using search engines, we conduct a comprehensive user study. We show that for questions regarding medical treatment effectiveness, participants struggle to find the correct answer and are prone to overestimating treatment effectiveness. We analyze participants' demographic traits according to age and education level and show that this problem persists in all demographic groups. We then propose a semi-automatic machine learning approach to find the correct answer to queries on medical treatment effectiveness as it is viewed by the medical community. The model relies on the opinions presented in medical papers related to the queries, as well as features representing their impact. We show that, compared to human behaviour, our method is less prone to bias. We compare various configurations of our inference model and a baseline method that determines treatment effectiveness based solely on the opinion of medical papers. The results bolster our confidence that our approach can pave the way to developing automatic bias-free tools that can help mediate complex health related content to users.

The use of autonomous robots is appealing for tasks, which are dangerous to humans. Autonomous robots might fail to perform their tasks since they are susceptible to varied sorts of faults such as point and contextual faults. Not all faults can be known in advance, and hence, anomaly detection is required. In this paper, we present an online data-driven anomaly detection approach (ODDAD) for autonomous robots. ODDAD is suitable for the dynamic nature of autonomous robots since it declares a fault based only on data collected online. In addition, it is unsupervised, model free and domain independent. ODDAD proceeds in three steps: data filtering, attributes grouping based on dependency between attributes and outliers detection for each group. Above a calculated threshold, an anomaly is declared. We empirically evaluate ODDAD in different domains: commercial unmanned aerial vehicles (UAVs), a vacuum-cleaning robot, a high-fidelity flight simulator and an electrical power system of a spacecraft. We show the significance and impact of each component of ODDAD . By comparing ODDAD to other state-of-the-art competing anomaly detection algorithms, we show its advantages.

As computerized agents have become more and more common, e-commerce has become a major candidate for incorporation of automated agents. Thus, it is vital to understand how people design agents for online markets and how their design changes over time. This, in turn, will enable a better design of agents for these environments. We focus on the design of trading agents for bilateral negotiations with unenforceable agreements. In order to simulate this environment we conducted an experiment with human subjects who were asked to design agents for a resource allocation game. The subjects' agents participated in several tournaments against each other and were given the opportunity to improve their agents based on their performance in previous tournaments. Our results show that, indeed, most subjects modified their agents' strategic behavior with the prospect of improving the performance of their agents. However, their average score significantly decreased throughout the tournaments and became closer to the equilibrium agents' score. In particular, the subjects modified their agents to break more agreements throughout the tournaments, while also increasing their means of protection against deceiving agents.

In many domains, an autonomous agent needs to reliably predict the distribution of behaviors of a population rather than the behavior of a single agent. For example, when playing the ultimatum game against several unknown opponents from a large known population, the agent can perform better by extracting its best-response strategy based on the distribution of the acceptance value in that population. In this paper, we demonstrate the efficacy of Peer-Designed-Agents (PDAs) for producing a distribution of behaviors that highly resembles the distribution of actual behaviors of a specific population of interest. This is obtained through extensive experiments with more than 700 different individuals and 132 PDAs, using eight game variants from three different domains and two different statistical tests. The analysis of the results demonstrates that PDAs' technology is an effective means for generating a reliable distribution of behaviors of a population of interest, as long as the similarity between the group of PDAs' developers and the latter population is sufficiently high. Moreover, a comprehensive comparison with the results of Elicited-Strategy-Agents (ESAs) shows that there is much more to PDA technology than simply an expression of strategy.

There is inconclusive evidence whether practicing tasks with computer agents improves people's performance on these tasks. This paper studies this question empirically using extensive experiments involving bilateral negotiation and three-player coordination tasks played by hundreds of human subjects. We used different training methods for subjects, including practice interactions with other human participants, interacting with agents from the literature, and asking participants to design an automated agent to serve as their proxy in the task. Following training, we compared the performance of subjects when playing state-of-the-art agents from the literature. The results revealed that in the negotiation settings, in most cases, training with computer agents increased people's performance as compared to interacting with people. In the three player coordination game, training with computer agents increased people's performance when matched with the state-of-the-art agent. These results demonstrate the efficacy of using computer agents as tools for improving people's skills when interacting in strategic settings, saving considerable effort and providing better performance than when interacting with human counterparts.

The design of automated negotiators has been the focus of abundant research in recent years. However, due to difficulties involved in creating generalized agents that can negotiate in several domains and against human counterparts, many automated negotiators are domain specific and their behavior cannot be generalized for other domains. Some of these difficulties arise from the differences inherent within the domains, the need to understand and learn negotiators’ diverse preferences concerning issues of the domain, and the different strategies negotiators can undertake. In this paper we present a system that enables alleviation of the difficulties in the design process of general automated negotiators termed Genius, a General Environment for Negotiation with Intelligent multi‐purpose Usage Simulation. With the constant introduction of new domains, e‐commerce and other applications, which require automated negotiations, generic automated negotiators encompass many benefits and advantages over agents that are designed for a specific domain. Based on experiments conducted with automated agents designed by human subjects using Genius we provide both quantitative and qualitative results to illustrate its efficacy. Finally, we also analyze a recent automated bilateral negotiators competition that was based on Genius. Our results show the advantages and underlying benefits of using Genius and how it can facilitate the design of general automated negotiators.

Persuasion is a common social and economic activity. It usually arises when conflicting interests among agents exist, and one of the agents wishes to sway the opinions of others. This paper considers the problem of an automated agent that needs to influence the decision of a group of self-interested agents that must reach an agreement on a joint action. For example, consider an automated agent that aims to reduce the energy consumption of a nonresidential building, by convincing a group of people who share an office to agree on an economy mode of the air-conditioning and low light intensity. In this paper we present four problems that address issues of minimality and safety of the persuasion process. We discuss the relationships to similar problems from social choice, and show that if the agents are using Plurality or Veto as their voting rule all of our problems are in P. We also show that with K-Approval, Bucklin and Borda voting rules some problems become intractable. We thus present heuristics for efficient persuasion with Borda, and evaluate them through simulations.

In this paper, we investigate the usefulness of peer-designed agents (PDAs) as a turn-key technology for enhancing parking simulations. The use of PDAs improves the system's ability to capture the dynamics of the interaction between individuals in the system, each theoretically exhibiting a different strategic behavior. Furthermore, since people in general are inherently rational and computation bounded, simulating this domain becomes even more challenging. The advantage of PDAs in this context lies in their ability to reliably simulate a large pool of human individuals with diverse strategies and goals. We demonstrate the efficacy of the proposed method by developing a large-scale simulation system for the parking space search domain, which plays an important role in urban transport systems. The system is based on 34 different parking search strategies. Most of these strategies are substantially different from synthetic strategies that are used in prior literature. A quantitative analysis of the PDAs indicates that they reliably capture their designers' real-life strategies. Finally, we demonstrate the usefulness of PDA-based parking space search simulation by utilizing it to evaluate four different information technologies that are of increasing use in recent years.

In May 2011, we organized the Second International Automated Negotiating Agents Competition (ANAC2011) in conjunction with AAMAS 2011. ANAC is an international competition that challenges researchers to develop a successful automated negotiator for scenarios where there is incomplete information about the opponent. One of the goals of this competition is to help steer the research in the area of bilateral multi-issue negotiations, and to encourage the design of generic negotiating agents that are able to operate in a variety of scenarios. Eighteen teams from seven different institutes competed in ANAC2011. This chapter describes the participating agents and the setup of the tournament, including the different negotiation scenarios that were used in the competition. We report on the results of the qualifying and final round of the tournament.

This paper presents an in-depth analysis and the key insights gained from the Second International Automated Negotiating Agents Competition (ANAC 2011). ANAC is an international competition that challenges researchers to develop successful automated negotiation agents for scenarios where there is no information about the strategies and preferences of the opponents. The key objectives of this competition are to advance the state-of-the-art in the area of practical bilateral multi-issue negotiations, and to encourage the design of agents that are able to operate effectively across a variety of scenarios. Eighteen teams from seven different institutes competed. This paper describes these agents, the setup of the tournament, including the negotiation scenarios used, and the results of both the qualifying and final rounds of the tournament. We then go on to analyse the different strategies and techniques employed by the participants using two methods: (i) we classify the agents with respect to their concession behaviour against a set of standard benchmark strategies and (ii) we employ empirical game theory (EGT) to investigate the robustness of the strategies. Our analysis of the competition results allows us to highlight several interesting insights for the broader automated negotiation community. In particular, we show that the most adaptive negotiation strategies, while robust across different opponents, are not necessarily the ones that win the competition. Furthermore, our EGT analysis highlights the importance of considering metrics, in addition to utility maximisation (such as the size of the basin of attraction), in determining what makes a successful and robust negotiation agent for practical settings.

The ability to reliably represent and replicate choices people make is crucial for building accurate models of day-today situations. The fact that people are inherently rationally-and computationally-bounded increases the difficulties in designing such simulations. This paper builds on the use of peer-designed agents (PDAs) - computer agents developed by people - to show their effectiveness in generating a variety of strategies and behaviors and in alleviating the simulation and behavior analysis of systems populated by human individuals with diverse strategies. The paper synthesizes the PDA-based simulation components and ideas that appear in recent PDAs literature into a cohesive simulation design, and reports a set of experiments aiming at validating the ability of PDA-based simulations to exhibit realistic behavior both in the individual agent and the system levels. The validation of individuals' ability to reliably capture their strategies into PDAs is quantitative, relying on four games from different domains. The domains vary in aspects such as game complexity and the environment dynamics. The applicability of the PDA-based approach in the system level is evaluated using a large scale experiment involving 34 PDAs, each designed by a different person. All in all, the set of strategies obtained by means of PDAs is substantially richer and more varied in comparison to the limited sets of strategies used in prior multi-agent simulation studies.

Research on automated negotiators has flourished in recent years. Among the important issues considered is how these automated negotiators can proficiently negotiate with people. To validate this, many experimentations with people are required. Nonetheless, conducting experiments with people is timely and costly, making the evaluation of these automated negotiators a very difficult process. Moreover, each revision of the agent's strategies requires to gather an additional set of people for the experiments. In this paper we investigate the use of Peer Designed Agents (PDAs) computer agents developed by human subjects as a method for evaluating automated negotiators. We have examined the negotiation results and its dynamics in extensive simulations with more than 300 human negotiators and more than 50 PDAs in two distinct negotiation environments. Results show that computer agents perform better than PDAs in the same negotiation contexts in which they perform better than people, and that on average, they exhibit the same measure of generosity towards their negotiation partners. Thus, we found that using the method of peer designed negotiators embodies the promise of relieving some of the need for people when evaluating automated negotiators.

Motivated by the challenges of bilateral negotiations between people and automated agents we organized the first automated negotiating agents competition (ANAC 2010). The purpose of the competition is to facilitate the research in the area bilateral multi-issue closed negotiation. The competition was based on the Genius environment, which is a General Environment for Negotiation with Intelligent multi-purpose Usage Simulation. The first competition was held in conjunction with the Ninth International Conference on Autonomous Agents and Multiagent Systems (AAMAS-10) and was comprised of seven teams. This paper presents an overview of the competition, as well as general and contrasting approaches towards negotiation strategies that were adopted by the participants of the competition. Based on analysis in post–tournament experiments, the paper also attempts to provide some insights with regard to effective approaches towards the design of negotiation strategies.

The development of proficient automated agents has flourished in recent years, yet making the agents interact with people has still received little attention. This is mainly due to the unpredictable nature of people and their negotiation behavior, though complexity and costs attached to experimentation with people, starting from the design and ending with the evaluation process, is also a factor. Even so, succeeding in designing proficient automated agents remains an important objective. In recent years, we have invested much effort in facilitating the design and evaluation of automated agents interacting with people, making it more accessible to researchers. We have created two distinct environments for bargaining agents, as well as proposing a novel approach for evaluating agents. These are key factors for making automated agents become a reality rather than remain theoretical.

Coordination in mixed agent-human environments is an important, yet not a simple, problem. Little attention has been given to the issues raised in teams that consist of both computerized agents and people. In such situations different considerations are in order, as people tend to make mistakes and they are affected by cognitive, social and cultural factors. In this paper we present a novel agent designed to proficiently coordinate with a human counterpart. The agent uses a neural network model that is based on a pre-existing knowledge base which allows it to achieve an efficient modeling of a human's decisions and predict their behavior. A novel communication mechanism which takes into account the expected effect of communication on the other member will allow communication costs to be minimized. In extensive simulations involving more than 200 people we investigated our approach and showed that our agent achieves better coordination when involved, compared to settings in which only humans or another state-of-the-art agent are involved.

Mediation is an important paradigm for dispute resolution which dates back to Ancient Greece. The goal is to reach an agreement, i.e. a mutually accepted solution, without resorting to a struggle. Yet, when people negotiate they demonstrate bounded rationality in their actions and diversity in their behaviors. This increases the difficulty to design automated mediators. To be successful, the mediator must take this into account and propose solutions deemed relevant, otherwise it will lose the focus and trust of the negotiators. We introduce AniMed* – an automated vivid animated mediator, incorporated with a novel proposal generation strategy, aimed to increase the social benefit of the negotiating parties. We will display the negotiation system and allow people to interact “face-to-face” with the automated mediator on various domains.

Autonomy requires robustness. The use of unmanned (autonomous) vehicles is appealing for tasks which are dangerous or dull. However, increased reliance on autonomous robots increases reliance on their robustness. Even with validated software, physical faults can cause the controlling software to perceive the environment incorrectly, and thus to make decisions that lead to task failure. We present an online anomaly detection method for robots, that is light-weight, and is able to take into account a large number of monitored sensors and internal measurements, with high precision. We demonstrate a specialization of the familiar Mahalanobis Distance for robot use, and also show how it can be used even with very large dimensions, by online selection of correlated measurements for its use. We empirically evaluate these contributions in different domains: commercial Unmanned Aerial Vehicles (UAVs), a vacuum-cleaning robot, and a high-fidelity flight simulator. We find that the online Mahalanobis distance technique, presented here, is superior to previous methods.