This work investigates why recurrent neural networks (RNNs) tend to learn phonological patterns that are unattested or dispreferred by humans. Specifically, we explore the hypothesis that their over-generation is caused by their excess expressive capacity - they are beyond the limited complexity class that contains the set of attested phonological patterns. We compared these over-expressive RNNs against the weaker convolutional neural networks (CNNs) on a battery of string recognition tasks. We find that the expressivity of a model's architecture does not predict the string classes that it excels at recognizing. Instead, we suggest that CNNs' position-invariant biases better explain their successes in our experiment.
This paper investigates the degree of semantic precision in derivational processes, a crucial yet underexplored aspect of word-formation semantics that influences our understanding of compositionality, competition, and polysemy in derivation. The study focuses on the case of verb-to-noun derivation in French, using both manual analysis and computational techniques to determine the level of semantic granularity that best captures the semantic properties of derivational processes. A sample of 2,190 derived nouns formed with 30 deverbal processes is manually analyzed, distinguishing between ontological and relational semantic types, with three levels of granularity applied to each. The various combinations of semantic types and granularities are tested against computational representations of derivational processes generated from distributional semantic models. The results indicate that not only relational but also ontological types participate in the semantics of derivational processes, and that derivation operates at a fine level of semantic granularity. More broadly, the study illustrates how expert analysis of large datasets can be fruitfully associated with computational methods to address key research questions in theoretical linguistics. It also advocates for a close integration of lexical semantics and derivational morphology in the study of complex words.
This study investigates directional constraints on diachronic tone chain shifts and their applicability to non-chain shifts. Twenty-eight chain shift changes from 12 Sino-Tibetan, Kra-Dai, Austronesian, and Otomanguean languages were compared to a sample of 118 non-chain shifts (including merger) from 54 Asian tone languages. Significant overlap was found: the reported chain shift changes were also the most frequently occurring changes in non-chain shifts. Recurring patterns emerged in both chain shifts and non-chain shifts: (1) tonal alignment slides rightward, with effects on contour shape, f0 excursion and f0 onset height; and (2) falling tones become higher while rising tones lower. These crosslinguistic diachronic trends show parallels to tone variation in connected speech, which suggests that the direction of diachronic tone change is phonetically grounded, constrained by articulatory and cognitive biases in speech production and perception.
Instance models have been successfully applied to a range of problems including memory, language, attention, learning, action, decision-making, and categorization. According to instance theory, the individual experience constitutes the fundamental unit of knowledge and knowledge of the general emerges during parallel retrieval from memory. Until recently, applications of instance theory to the problem of language were constrained to small and contrived laboratory experiments. However, recent advances in large-scale computational modeling have allowed the approach to be applied at scale to the large and complicated problem of natural language. With those demonstrations now in hand, we argue that the framework can present an articulate mechanistic underbelly to usage-based theories of language that highlights the role of specific language experience in general language behavior. Overall, this article argues that instance memory models provide an opportunity to gain insight into and deepen our understanding of language as a dynamic and contextually embedded process, serving to bridge the gap between cognitive psychology and the language sciences.
Recently, deep learning models have increasingly been used in cognitive modelling of language. This study asks whether deep learning can help us to better understand the learning problem that needs to be solved by speakers, above and beyond linear methods. We utilize the Discriminative Lexicon Model introduced by Baayen and colleagues, which models comprehension and production with mappings between numeric form and meaning vectors. While so far, these mappings have been linear (linear discriminative learning; LDL), in the present study we replace them with deep dense neural networks (deep discriminative learning; DDL). We find that DDL affords more accurate mappings for large and diverse datasets from English and Dutch, but not necessarily for Estonian and Taiwan Mandarin. DDL outperforms LDL in particular for words with pseudo-morphological structure such as chol + er. Applied to average reaction times, we find that DDL is outperformed by frequency-informed linear mappings (FIL). However, DDL trained in a frequency-informed way ("frequency-informed" deep learning; FIDDL) substantially outperforms FIL. Finally, while linear mappings can very effectively be updated from trial-to-trial to model incremental lexical learning, deep mappings cannot do so as effectively. At present, both linear and deep mappings are informative for understanding language.
