For gathering data on syntax-prosody relations, it has been unclear how to proceed experimentally. This is especially so for complex syntactic structures, such as the doubly center-embedded relative clause construction, which is syntactically well-formed but notoriously difficult to parse. These complex sentences can be especially revealing theoretically but cannot easily be elicited from speakers by presentation of picture choices or written preambles. While acknowledging that it may not be ideal, many studies of these and other complex constructions have resorted to a simple methodology in which written target sentences are read aloud. A basic methodological decision is then whether or not to permit (or encourage) the reader to preview the text before voicing it aloud. The results of reading with preview and of reading ‘cold’ without preview can both be informative, but in different ways. Reading without preview taps on-line performance, which can reveal possible syntactic/semantic expectations, and may shed light on the implicit prosody of silent reading. Reading with preview should provide a better window on prosodic competence: the reader’s inherent knowledge of the prosody/syntax alignment principles of the grammar. However, we maintain that previewing by reading aloud, as in the Double Reading design that we report on here, can be more informative of prosodic competence than the typical silent reading preview.

AbstractFirst I would like to acknowledge the contributions of my collaborators, especially my colleague William Sakas, and our graduate students. We are all part of the CUNY Computational Language Acquisition Group (CUNY-CoLAG), whose mission is the computational simulation of syntax acquisition. We have created a large domain of languages, similar to natural languages though simplified, which we use to test the accuracy and speed of different models of child language acquisition.

Recent challenges to Chomsky's poverty of the stimulus thesis for language acquisition suggest that children's primary data may carry "indirect evidence" about linguistic constructions despite containing no instances of them. Indirect evidence is claimed to suffice for grammar acquisition, without need for innate knowledge. This article reports experiments based on those of Reali and Christiansen (2005), who demonstrated that a simple bigram language model can induce the correct form of auxiliary inversion in certain complex questions. This article investigates the nature of the indirect evidence that supports this learning, and assesses how reliably it is available. Results confirm the original finding for one specific sentence type but show that the model's success is highly circumscribed. It performs poorly on inversion in related constructions in English and Dutch. Because other, more powerful statistical models have so far been shown to succeed only on the same limited subset of cases as the bigram model, it remains to be seen whether stimulus richness can be substantiated more generally.

Language learners with insufficient access to negative evidence about what is not in their target language must rely on the Subset Principle (SP), or some other similar conservative learning strategy, in order to avoid overgeneration. Recent attempts to incorporate such a strategy into psychologically realistic models of syntax acquisition have revealed two severe problems: SP application appears to demand computational resources that exceed those of children; and SP causes undergeneration failures if learning is incremental. We present a representational scheme for the domain of grammars which can alleviate both problems, and we report simulation data showing how it can best be employed in a learning model. Implementation Challenges Because language learners receive little information about non-sentences of their target language (Marcus, 1993), any model of natural language syntax acquisition must have some means of avoiding or minimizing overgeneration. The learning mechanism (LM) must be conservative: other things being equal, the grammar hypothesis it adopts must be the one that fits the positive input most snugly. This general principle has been cast as the Subset Principle in studies of syntax acquisition grounded in generative linguistics (Berwick, 1985; Manzini & Wexler, 1987). It is also a close relation of the domain-general size principle of Bayesian learning theory (Tenenbaum & Griffiths, 2001). For convenience here we will refer to this conservative tendency as the Subset Principle (SP) but leaving open the existence of many varied implementations of it. Our concern is a duo of recently uncovered practical problems that must be addressed by any such implementation if it is intended as a contribution to a psychological model of how children acquire syntax. As noted in Fodor & Sakas (2005), one problem is that rigorous application of SP appears to demand an undue share of the on-line computational resources that can reasonably be ascribed to a pre-school child. The second problem is that under some familiar learning regimes, SP becomes over-zealous and prevents convergence on the target grammar: without SP, learners are at risk of overgeneration errors, but with SP they are at risk of undergeneration errors. Thus despite its central importance, it is unclear whether SP (and/or its close relations in other frameworks, including statistical learning models) can be successfully incorporated into psychologically faithful models of language acquisition. We illustrate these problems below in a specific modeling framework that has served in the past as our basis for simulation experiments comparing the efficiency of various acquisition tactics (Fodor & Sakas, 2004). The targets for learning are parameter-based grammars (Chomsky, 1981 et seq.). In parameter setting (‘triggering’) models, it is commonly assumed that LM has no memory for prior input sentences or for which grammars it entertained previously. It retains from its past experience only the knowledge that is encapsulated in its current grammar. Thus, in contrast to models that accumulate data and seek regularities in it, parameter setting is incremental, in the sense that LM receives target language sentences one at a time and decides, on the basis of each one, either to retain its current grammar hypothesis or to switch to a different one. Despite these specific properties, we believe that the points we raise here have bearing on a broad range of approaches to syntax acquisition. The implementation of SP is equally challenging, or more so, for other current learning models, and any advances that can be made may therefore benefit those other approaches as well. In this paper we argue that it is essential to augment in some way the severely restricted memory of incremental models, and we propose a novel representational scheme that allows LM to keep track of the domain of grammar hypotheses, and thereby alleviates both the problem of on-line computational resources and the undergeneralization problem. The Computational Resources Problem SP is a comparative criterion for grammar selection: whether it permits a grammar hypothesis to be adopted depends on what alternative hypotheses are available. Given input sentence i, LM should ideally adopt a grammar G such that the language L(G) includes i and has no proper subset L(G′) that includes i, where G′ is a possible grammar that has not been disconfirmed by prior input (if the model has knowledge of that; see below). But how can LM know which grammar satisfies these criteria? It appears that LM must have the ability to identify grammars that license an arbitrary sentence i, and moreover that it must have exhaustive knowledge of all (non-disconfirmed) grammars that license i, so that it can compare them against each other to ensure that it does not unwittingly adopt one that is prohibited by the existence of a less inclusive one. Thus, when LM’s current grammar fails on an input i and a new grammar must be adopted, LM has three tasks to do. Task A: Find a new grammar hypothesis G which does license i. Task B: Identify all other grammars that license i (in order to be able to check for subset relations as in Task C). Task C: Check whether any other grammar that licenses i generates a subset of L(G). Task A has proved to be a cumbersome problem for syntax acquisition models. It is not always obvious by inspection of an input word string what grammar might have generated it. Various strategies which start from the current grammar and amend it (e.g., reset one parameter at a time; reset only incorrect parameters) have been found to be inadequate because, for example, it is often unclear which parameters are incorrect. Recent models typically undertake extensive trial and error, selecting a grammar and then testing to see whether it will parse i (e.g., Gibson & Wexler, 1994; Clark, 1992; Yang, 2002). The models that we have developed use the parsing routines instead to identify needed changes to the current grammar (Sakas & Fodor, 2001). However, this technique has its limits. It can reliably identify one grammar that generates i, but not more than one without exceeding standardly accepted limits on the capacity of the human parsing mechanism. Task B (identifying all grammars compatible with i) is a challenge of a higher order. The natural language domain is highly ambiguous, with most sentence types compatible with multiple grammars (Clark, 1989). It is also a very large search space, possibly on the order of billions of grammars (2 for n independent binary parameters), so the workload would be prohibitive if indeed every grammar must be checked whenever LM is considering adopting a new one. It is clearly beyond the bounds of psychological plausibility to suppose that a child runs a billion parse tests, each with a different grammar, on a single input sentence to see which grammars succeed. To solve this problem, a completely different approach to SP is required which does not require exhaustive knowledge of all grammars that license i, as we discuss below. Task C (discovering subset relations between grammars that license i) might be achieved by comparing languages (sets of sentences) on-line, but this too would exceed plausible computational resources. An alternative approach would be to assume that LM is equipped with prior information as to which languages are subsets of which others. Ideally, these subset relations between languages would be transparently reflected in formal relations between their grammars, so that LM could simply inspect two grammars to find out whether one generates a subset of the other. This was proposed by Manzini & Wexler (1987), who suggested that each parameter has a default value and a marked value (notated 0 and 1 respectively) and that subset relations between grammars are due exclusively to these values: for any pair of grammars differing with respect to the value of a parameter P, the language with value 0 for P is a proper subset of the language with value 1 for P; and no other subset-superset relations hold between any grammars in the domain. We have called this the Simple Defaults Model (Fodor & Sakas, 2005). If it were true of natural languages, it would strongly limit the number of subset relations in the domain, thus reducing the scale of Task C. And it would provide LM with a trivially easy way to identify all the subsets of a language L(G): they would be all and only those languages whose grammars differ from G by having value 0 for one or more parameters for which G has value 1. Unfortunately, it seems that this optimal situation does not obtain in the case of natural languages. For our parameter-setting simulation experiments we have created a domain of 3,072 artificial languages, defined by 13 syntactic parameters and designed to be as much like real natural languages as possible despite necessary simplifications. In this domain the Simple Defaults Model fails. A high proportion (over 42%) of the subset relations that hold between grammars are not predictable from the subset values of individual parameters; they are due instead to interactions, often quite unruly, among two or more parameters. Therefore, any SP-implementation based on the Simple Defaults Model would under-report the subsets a language has, and would fail to protect LM against overgeneration errors. Simulation data confirm this expectation; we observe 64% failures for a model that performs without error when supplied with full information about subset relations. Perhaps other linguistic theories might offer better ways of predicting subset relations between languages based on their grammars, but none is known at present and in fact there are good reasons to suspect that the relationship between grammars and the languages they generate is bound to be disorderly: a small change in a grammar can completely change the set of sentences (word strings) it generates, and word strings generated by quite d

Abstract Modest experimental findings support the general moral that this chapter is tempted to draw on the basis of informal judgments of written and spoken sentences. That is, acceptability judgments on written sentences are not purely syntax-driven; they are not free of prosody even though no prosody exists in the stimulus. This has a useful result for the conduct of syntactic research: more widespread use must be made of spoken sentences for achieving syntactic well-formedness judgments. The ideal way of presentation gives both written and auditory versions of the sentence, to reduce perceptual memory errors while making sure that the sentence is being judged on the basis of the prosody intended.

Our research group has built a research environment for testing models of syntactic parameter setting. We have created a domain of 3,072 languages with up to 1,432 sentences in each, defined by a set of Universal Grammar (UG) rules and 13 (so far) binary parameters. The languages resemble human languages in many respects, though they are considerably simpler (details below). Each of these languages is designated in turn as the target for acquisition and its sentences are input to a learning algorithm. We measure whether learning is successful (is the target reliably attained?) and how long it takes (how many input sentences the learner consumes before arriving at the target grammar). Our goal in preparing this rich testing environment was to further the search for a credible psycho-computational model of syntactic parameter setting. That sounds very grand, but what it means is just: a model that is psychologically realistic, precisely specified, compatible with linguistic principles, and as reliably successful as children are. This has proven remarkably elusive. Individual grammars are defined by their parameter values, so acquiring a language consists in identifying the parameter values that license it. When the principles and parameters (P&P) theory of grammars was first proposed (Chomsky, 1981), it was hailed as a sweeping solution to problems that had

When we hear or read a sentence, we are aware more or less instantaneously of what it means . Our minds compute the meaning somehow, on the basis of the words that comprise the sentence. But the words alone are not enough . The sentence meanings we establish are so precise that they could not be arrived at by just combining word meanings haphazardly . A haphazard word combiner could misunderstand (1) as meaning that all bears love; it could interpret (2) as meaning that pigs fly and rabbits can't .

The argument from the poverty of the stimulus as Pullum and Scholz define it (their APS) is undeniably true, given that all language learners acquire the ability to generate more sentences of the target language than they have heard. Uniformity across learners with respect to the additional sentences they project suggests that grammar induction is guided by general principles, which must be innate. What remains to be established is exactly which sentences can be projected on the basis of which others. The details of this are important to linguistic theory and to the psycho-computational modelling of natural language acquisition. They are not of great significance to the generic issue of nativism versus empiricism, except that they may clarify the extent to which the innate knowledge in question is specific to language. The argument for linguistic nativism appears to be solidly supported by the distinctive patterns of generalization that learners adopt in the absence of systematic negative evidence (a limitation that Pullum and Scholz exclude from APS). We argue that innate knowledge of how to represent natural language facts is necessary in order for learners to extract from their input the information that it does contain. Pullum and Scholz themselves rely on Universal Grammar in just this role when they make specific suggestions as to how learners arrive at the right generalizations.

How much work does it take to acquire a human language? For most adults, the acquisition of a new language is a slow and effortful process. But what if one has the right learning equipment, as children evidently do? For first language learners most of the work is done in five or six years. Our research goal is to find out what goes on in those few years. To what extent does it involve the use of special-purpose computational systems that adults lack? What do the learning routines do that is so difficult for the human brain to simulate later in life?

The human sentence processing device sometimes makes errors, and when it does, it can sometimes correct them. This much is generally agreed, though opinions differ with respect to how and why the errors occur. In this paper we are concerned with the process of recovery from garden paths in sentence processing. A garden path occurs when the parser makes an error in assigning structure to the input word string but is nevertheless able to continue integrating some subsequent words into the structure that it has constructed for the sentence so far (the current partial phrase marker, or CPPM). Recognition that a garden path has occurred comes from the subsequent discovery that there is a word in the input string which does not fit into the CPPM. This word is the error signal, or symptom, that reveals the existence of the earlier error of analysis. The parser’s task is to discover the nature of the problem and put it right if possible. The input may actually be ungrammatical, in which case nothing can be done. But the parser must also consider the possibility that it is the analysis that is at fault: that some aspect of the CPPM prior to the symptom is incorrect. Recovery from a garden path consists in finding an alternative analysis which fits the initial portion of the sentence and also accommodates the symptom and later words.

The symptom of a garden path in sentence processing is an apparent anomaly in the input string. This anomaly signals to the parser that an error has occurred, and provides cues for how to repair it. Anomaly detection is thus an important aspect of sentence processing. In the present study, we investigated how the parser responds to unambiguous sentences that contain syntactic anomalies and pragmatic anomalies, examining records of eye movement during reading. While sensitivity to the two kinds of anomaly was very rapid and essentially simultaneous, qualitative differences existed in the patterns of first-pass reading times and eye regressions. The results are compatible with the proposal that syntactic information and pragmatic information are used differently in garden-path recovery.

Three experiments were conducted to investigate the relative timing of syntactic and pragmatic anomaly detection during sentence processing. Experiment 1 was an eye movement study. Experiment 2 employed a dual-task paradigm with compressed speech input, to put the processing routines under time pressure. Experiment 3 used compressed speech input in an anomaly monitoring task The outcomes of these experiments suggest that there is little or no delay in pragmatic processing relative to syntactic processing in the comprehension of unambiguous sentences. This narrows the possible explanations for any delays that are observed in the use of pragmatic information for ambiguity resolution.

Janet Fodor, Ivan Sag : A traceless account of extraction phenomena Trace theory is an unquestioned part of transformational grammars and Governement and Binding theory. We show that theory external motivation for positing phonetically empty elements is in fact minimal, and no empirical evidence (syntactic, phonetic or psycholinguistic) can be found in favor of them in extraction contexts. We present an alternative view in Head-driven Phrase Structure Grammar that can formalize the notion of a missing complement via syntactic features without positing a trace in a syntactic tree.

We propose that, for the human parser, recovery from garden paths consists in repairing the structure built so far, rather than reparsing the input. The difficulty of a repair is attributable not to the cost of effecting the structural alterations but to the cost of deducing which alterations are needed. The parser must diagnose its error in order to correct it. The error is signaled by an input word that is incompatible with the current structure; this is the symptom from which the diagnosis must be made. If the error is transparently clear from the nature of the symptom, recovery is easy; but sometimes the necessary reasoning is obscure, and then the diagnosis is unsuccessful and the garden path persists. Unlike other repair models, the diagnosis model needs no special mechanism for revising garden path analyses. The garden path recovery device is the same machine as the first-pass parser, merely set into emergency mode. When faced with a breakdown the parser does not stop its normal activities and enter a new mode of reasoning to detect what went wrong. It simply continues to parse, attaching the problematic input item in the least ungrammatical way it can, despite the conflict with previously built structure. This conflict is productive; it provokes adjustments to the existing structure. In successful cases, one adjustment leads to another until a stable state is reached, at which point the original error will have been eliminated. Examples suggest that the parser gives more weight to syntatctic than to pragmatic acceptability; only a syntactic clash between the input and the existing structure sets the adjustment process in motion.