The interpretation of other agents as intentional actors equipped with mental states has been connected to the attribution of rationality to their behavior. But a workable definition of “rationality” is difficult to formulate in complex situations, where standard normative definitions are difficult to apply. In this study, we explore a notion of rationality based on the idea of evolutionary fitness. We ask whether agents that are more adapted to their environment are, consequently, perceived as more rational and intentional. We created a 2-D virtual environment populated with autonomous virtual agents, each of which behaves according to a built-in program equipped with simulated perception, memory, and decision making. We then introduced a process of simulated evolution that pressured the agents’ programs toward behavior more adapted to the simulated environment. We showed these agents to human subjects in 2 experiments, in which we respectively asked them to judge their intelligence and to dynamically estimate their “mental states.” The results confirm that subjects construed evolved agents as more intelligent, and judged evolved agents’ mental states more accurately, relative to nonevolved agents. These results corroborate a view that the interpretation of agent behavior is connected to a concept of rationality based on the apparent fit between an agent’s actions and its environment.

Inferring the mental states of other agents, including their goals and intentions, is a central problem in cognition. A critical aspect of this problem is that one cannot observe mental states directly, but must infer them from observable actions. To study the computational mechanisms underlying this inference, we created a two-dimensional virtual environment populated by autonomous agents with independent cognitive architectures. These agents navigate the environment, collecting “food” and interacting with one another. The agents’ behavior is modulated by a small number of distinct goal states: attacking, exploring, fleeing, and gathering food. We studied subjects’ ability to detect and classify the agents’ continually changing goal states on the basis of their motions and interactions. Although the programmed ground truth goal state is not directly observable, subjects’ responses showed both high validity (correlation with this ground truth) and high reliability (correlation with one another). We present a Bayesian model of the inference of goal states, and find that it accounts for subjects’ responses better than alternative models. Although the model is fit to the actual programmed states of the agents, and not to subjects’ responses, its output actually conforms better to subjects’ responses than to the ground truth goal state of the agents.

Perception of intentions and mental states in autonomous virtual agents Peter C. Pantelis, Steven Cholewiak, Paul Ringstad, Kevin Sanik, Ari Weinstein, Chia-Chien Wu, Jacob Feldman (petercp@eden.rutgers.edu, jacob@ruccs.rutgers.edu) Departments of Psychology, Center for Cognitive Science, Rutgers University-New Brunswick 152 Frelinghuysen Road, Piscataway, NJ 08854 USA Abstract 1995; Johnson, 2000). But the adult capacity to understand animate motion in terms of intelligent behavior has been re- searched less. Computational approaches to the problem of intention estimation are still scarce (Baker, Tenenbaum, & Saxe, 2006; Feldman & Tremoulet, 2008), in part because of the difficulty in specifying the problem in computational terms. Almost without exception, video stimuli used in past ex- periments in this area have consisted of hand-crafted an- imations with motions chosen subjectively by the experi- menters in order to achieve particular psychological impres- sions (Porter & Susman, 2000). This makes it difficult to in- vestigate the way subjects estimate intentionality, because the object of the estimation procedure—the actual mental state of the agent under observation—does not actually exist. Our proposed solution to this problem is to indeed endow our stimuli agents with “minds,” which our subjects then attempt to “read.” Comprehension of goal-directed, intentional motion is an im- portant but understudied visual function. To study it, we created a two-dimensional virtual environment populated by independently-programmed autonomous virtual agents, which navigate the environment, collecting food and competing with one another. Their behavior is modulated by a small number of distinct “mental states”: exploring, gathering food, attacking, and fleeing. In two experiments, we studied subjects’ ability to detect and classify the agents’ continually changing men- tal states on the basis of their motions and interactions. Our analyses compared subjects’ classifications to the ground truth state occupied by the observed agent’s autonomous program. Although the true mental state is inherently hidden and must be inferred, subjects showed both high validity (correlation with ground truth) and high reliability (correlation with one an- other). The data provide intriguing evidence about the factors that influence estimates of mental state—a key step towards a true “psychophysics of intention.” Keywords: animate motion perception; theory of mind; inten- tionality; action understanding; goal inference. Introduction Comprehension of the goals and intentions of other intelligent agents is an essential aspect of cognition. Motion is an espe- cially important cue to intention, as vividly illustrated by the famous short film by Heider and Simmel (1944). The “cast” of this film consists only of two triangles and a circle, but the motions of these simple geometrical figures are universally interpreted in terms of dramatic narrative. Indeed, it is practi- cally impossible to understand many naturally occurring mo- tions without comprehending the intentions that helped cause them: a person running is interpreted as trying to get some- where; a hand lifting a Coke can is automatically understood as a person intending to raise the can, not simply as two ob- jects moving upwards together (Mann, Jepson, & Siskind, 1997). Much of the motion in a natural environment—and certainly some of the most behaviorally important motion— is caused by other agents, and is impossible to understand except in terms of how and why they might have caused it. Human subjects readily attribute mentality and goal- directedness to moving objects as a function of properties of their motion (Tremoulet & Feldman, 2000), and in par- ticular on how that motion seems to relate to the motion of other agents and objects in the environment (Blythe, Todd, & Miller, 1999; Dittrich & Lea, 1994; Gao, McCarthy, & Scholl, 2010; Pantelis & Feldman, 2010; Tremoulet & Feld- man, 2006; Zacks, Kumar, Abrams, & Mehta, 2009). The broad problem of attributing mentality to others has received a great deal of attention in the philosophical literature (of- ten under the term mindreading), and has been most widely studied in infants and children (Gelman, Durgin, & Kaufman, A virtual environment of autonomous agents We developed a two-dimensional interactive virtual envi- ronment populated with autonomous virtual agents (Fig. 1). These agents (referred to as Independent Mobile Personali- ties, or IMPs), are simple but cognitively independent virtual robots, equipped with perception, planning, decision making, and goals. They move about in the virtual environment, in- teracting with each other, making intelligent though unpre- dictable decisions and taking steps to achieve simple goals. The IMPs are endowed with potentially distinct personalities and cognitive faculties, including variations in intelligence, memory, aggression, and strategy. The result is a complex, dynamic microcosm in which goal-directed behavior, and the perception thereof, can be studied in a controlled way. The inspiration is taken from the substantial literature on artificial life (Shao & Terzopoulos, 2007; Yaeger, 1994) in which inter- actions among virtual creatures have been extensively mod- eled. But unlike previous environments, our agents are cog- nitively complete, meaning that their behavior is entirely de- termined by autonomous decisions based on input they have received via their own senses, and are presented visually to subjects so that we may study how their intentions are inter- preted by observers. Our focus is on what can be understood from the IMPs’ motion alone; to this end, we depict the IMPs as triangles, so that they have clearly identifiable main axes and front ends, but otherwise minimal shape. Because we have direct access to the “actual” intentions and mental states of the agents—represented by a simple state variable in the autonomous program—we can compare this “ground truth”

This paper investigates the ability of sparse line drawings to depict 3D shape. We perform a study in which people are shown an image of one of twelve 3D objects depicted with one of six styles and asked to orient a gauge to coincide with the surface normal at many positions on the object's surface. The normal estimates are compared with each other and with ground truth data provided by a registered 3D surface model to analyze accuracy and precision. The paper describes the design decisions made in collecting a large data set (275,000 gauge measurements) and provides analysis to answer questions about how well people interpret shapes from drawings. Our findings suggest that people interpret certain shapes almost as well from a line drawing as from a shaded image, that current computer graphics line drawing techniques can effectively depict shape and even match the effectiveness of artist's drawings, and that errors in depiction are often localized and can be traced to particular properties of the lines used. The data collected for this study will become a publicly available resource for further studies of this type.