We propose the challenge of rapid task-solving in novel environments (RTS), wherein an agent must solve a series of tasks as rapidly as possible in an unfamiliar environment. An effective RTS agent must balance between exploring the unfamiliar environment and solving its current task, all while building a model of the new environment over which it can plan when faced with later tasks. While modern deep RL agents exhibit some of these abilities in isolation, none are suitable for the full RTS challenge. To enable progress toward RTS, we introduce two challenge domains: (1) a minimal RTS challenge called the Memory&Planning Game and (2) One-Shot StreetLearn Navigation, which introduces scale and complexity from real-world data. We demonstrate that state-of-the-art deep RL agents fail at RTS in both domains, and that this failure is due to an inability to plan over gathered knowledge. We develop Episodic Planning Networks (EPNs) and show that deep-RL agents with EPNs excel at RTS, outperforming the nearest baseline by factors of 2-3 and learning to navigate held-out StreetLearn maps within a single episode. We show that EPNs learn to execute a value iteration-like planning algorithm and that they generalize to situations beyond their training experience. algorithm and that they generalize to situations beyond their training experience.

Prior work decoding linguistic meaning from imaging data has been largely limited to concrete nouns, using similar stimuli for training and testing, from a relatively small number of semantic categories. Here we present a new approach for building a brain decoding system in which words and sentences are represented as vectors in a semantic space constructed from massive text corpora. By efficiently sampling this space to select training stimuli shown to subjects, we maximize the ability to generalize to new meanings from limited imaging data. To validate this approach, we train the system on imaging data of individual concepts, and show it can decode semantic vector representations from imaging data of sentences about a wide variety of both concrete and abstract topics from two separate datasets. These decoded representations are sufficiently detailed to distinguish even semantically similar sentences, and to capture the similarity structure of meaning relationships between sentences.

Recent research has placed episodic reinforcement learning (RL) alongside model-free and model-based RL on the list of processes centrally involved in human reward-based learning. In the present work, we extend the unified account of model-free and model-based RL developed by Wang et al. (2018) to further integrate episodic learning. In this account, a generic model-free “meta-learner” learns to deploy and coordinate among all of these learning algorithms. The meta-learner learns through brief encounters with many novel tasks, so that it learns to learn about new tasks. We show that when equipped with an episodic memory system inspired by theories of reinstatement and gating, the meta-learner learns to use the episodic and model-based learning algorithms observed in humans in a task designed to dissociate among the influences of various learning strategies. We discuss implications and predictions of the model.

Meta-learning agents excel at rapidly learning new tasks from open-ended task distributions; yet, they forget what they learn about each task as soon as the next begins. When tasks reoccur - as they do in natural environments - metalearning agents must explore again instead of immediately exploiting previously discovered solutions. We propose a formalism for generating open-ended yet repetitious environments, then develop a meta-learning architecture for solving these environments. This architecture melds the standard LSTM working memory with a differentiable neural episodic memory. We explore the capabilities of agents with this episodic LSTM in five meta-learning environments with reoccurring tasks, ranging from bandits to navigation and stochastic sequential decision problems.

Deep neural networks (DNNs) have achieved unprecedented performance on a wide range of complex tasks, rapidly outpacing our understanding of the nature of their solutions. This has caused a recent surge of interest in methods for rendering modern neural systems more interpretable. In this work, we propose to address the interpretability problem in modern DNNs using the rich history of problem descriptions, theories and experimental methods developed by cognitive psychologists to study the human mind. To explore the potential value of these tools, we chose a well-established analysis from developmental psychology that explains how children learn word labels for objects, and applied that analysis to DNNs. Using datasets of stimuli inspired by the original cognitive psychology experiments, we find that state-of-the-art one shot learning models trained on ImageNet exhibit a similar bias to that observed in humans: they prefer to categorize objects according to shape rather than color. The magnitude of this shape bias varies greatly among architecturally identical, but differently seeded models, and even fluctuates within seeds throughout training, despite nearly equivalent classification performance. These results demonstrate the capability of tools from cognitive psychology for exposing hidden computational properties of DNNs, while concurrently providing us with a computational model for human word learning.

In this paper we carry out an extensive comparison of many off-the-shelf distributed semantic vectors representations of words, for the purpose of making predictions about behavioural results or human annotations of data. In doing this comparison we also provide a guide for how vector similarity computations can be used to make such predictions, and introduce many resources available both in terms of datasets and of vector representations. Finally, we discuss the shortcomings of this approach and future research directions that might address them.

Complex interactions among the meanings of words are important factors in the function that maps word meanings to phrase meanings. Recently, compositional distributional semantics models (CDSM) have been designed with the goal of emulating these complex interactions; however, experimental results on the effectiveness of CDSM have been difficult to interpret because the current metrics for assessing them do not control for the confound of lexical information. We present a new method for assessing the degree to which CDSM capture semantic interactions that dissociates the influences of lexical and compositional information. We then provide a dataset for performing this type of assessment and use it to evaluate six compositional models using both co-occurrence based and neural language model input vectors. Results show that neural language input vectors are consistently superior to co-occurrence based vectors, that several CDSM capture substantial compositional information, and that, surprisingly, vector addition matches and is in many cases superior to purpose-built paramaterized models.

Consider the meanings of the following phrases: “red apple,” “red hair,” and “red state.” The meaning of the word “red” in each of these examples interacts with the meaning of the noun it modifies, applying a different color to the first two and a political affiliation to the third. This is an example of a common phenomenon in natural language in which the meaning of a whole expression is not derived from a simple sum of its parts, but is composed by interactions among their meanings. Semantic interactions have been acknowledged in the philosophical literature, notably by Frege [9], who asserted that a word in isolation is only an abstraction, and its precise meaning only assigned when it is placed in the context of a proposition. Further, semantic interaction is a detectable component of human language processing, as evidenced by the well documented phenomenon of sentential priming [8].

Causation, Force, and the Sense of Touch Phillip Wolff (pwolff@emory.edu) Department of Psychology, 36 Eagle Row Atlanta, GA 30322 USA Samuel Ritter (swritter@Princeton.edu) Department of Psychology, Green Hall Princeton, NJ 08540 USA Kevin J. Holmes (kjholmes@berkeley.edu) Department of Linguistics, 1203 Dwinelle Hall Berkeley, CA 94720 USA relevant today because they continue to be used in defense of probabilistic accounts of causation (e.g., Cheng, 1997; Cheng & Novick, 1991, 1992). In this paper, we report a set of findings that addresses Hume’s main criticism against force accounts of causation: specifically, that forces cannot be linked to internal or external sensory impressions. The criticism certainly holds in the case of the visual modality. However, once we consider the potential contributions of other senses, in particular touch, it becomes clear that people’s sensory experience is not as deficient as Hume (and many modern theorists) have claimed. According to what we will call the causal force hypothesis, people’s mental representation of causation is based on the feeling of force as understood through the sense of touch (see Fales, 1990; White, 2012). Abstract It is widely acknowledged that causation entails more than spatial-temporal contiguity or correlation, but efforts to specify that extra component of experience have been elusive. In this paper, we argue that the representation of causal relations is based on the feeling of force as understood through the sense of touch. Grounding causation in people’s sense of touch allows us to address long-standing challenges that have been raised against force-based approaches to causation. In support of our proposal, we report a series of experiments showing that the perception of causation is associated with the notion of force, as indicated by changes in people’s sensitivity to a physical force acting against their hand. We also show that when people associate correlations with force, they view those correlations as causal. Implications for understanding the origins of causal knowledge are discussed. Keywords: Causation; Causal perception; Force Perception; Haptics; Causal Induction; Abstract Concepts Perception of forces Introduction Several recent theories of causation have proposed that the mental representation of causation is based on the notion of force (Copley & Harley, 2014; Fales, 1990; Gardenfors, 2000; Mumford & Anjum, 2011; Hubbard & Ruppel, 2014; White, 2012; Warglien, Gardenfors, Westera, 2012; Wolff, 2007; Wolff, et al. 2010). These theories have provided explanations of how causal relations might be recognized from a single occurrence of an event (Ahn & Kalish, 2000; Bigelow, Ellis, & Pargetter, 1988; Wolff, 2007) as well as how different kinds of causal relationships might be related to one another (Talmy, 1988; see also Wolff, 2007; Wolff, et al., 2010). Despite these successes, there has been strong criticism of force-based accounts of causation (Cheng, 1997; Cheng & Novick, 1992; Schulz, Kushnir, & Gopnik, 2007; Woodward, 2007; Sloman, Barbey, & Hotalling, 2009). Arguably the most fundamental of these criticisms was initially made by Hume (1748/1975). He pointed out that the notion of force could not be linked to any internal or external sensory impression and that, therefore, forces could not be the basis for our mental representation of causation. He noted that after many repetitions of conjunctions of objects or events, people could develop an expectation that gives rise to a sense of power or force, but that this sense only emerged from statistical regularities, which were the only legitimate bases for inducing causation. Hume’s arguments remain Several lines of research have established that people are able to represent forces. These studies have shown that people are skilled at perceiving forces from the environment when those forces impinge directly on the skin. For example, Wheat, Salo, and Goodwin (2004) found a nearly linear relationship between participants’ estimates of a force acting on their fingers and the actual magnitude of the force. Panarese and Edin (2011) found that people are quite good at discriminating the direction of forces applied to the index finger. Of particular relevance to the induction of causation, several neuroimaging studies have reported evidence for the encoding of forces even in the absence of physical contact. For example, Keysers et al. (2004) observed activity in the somatosensory cortex not only when people were touched directly on their legs, but also when they observed other people being touched on their legs. Even more impressively, activity in the somatosensory cortex was observed when participants observed one inanimate object touch another inanimate object. Keysers et al.’s (2004) findings have been replicated and extended in several other studies (see Blakemore, Bristow, et al., 2005; Ebisch, et al., 2008). Indeed, the representation of forces through the visual modality is revealed in common everyday tasks. Many of us, for example, have had the experience of reaching for a suitcase or box and over-lifting it because we thought it was full when, in fact, it was empty. Such events presumably