We provide a mathematical argument showing that, given a representation of lexical items as functions (wavelets, for instance) in some function space, it is possible to construct a faithful representation of arbitrary syntactic objects in the same function space. This space can be endowed with a commutative non-associative semiring structure built using the second Renyi entropy. The resulting representation of syntactic objects is compatible with the magma structure. The resulting set of functions is an algebra over an operad, where the operations in the operad model circuits that transform the input wave forms into a combined output that encodes the syntactic structure. The action of Merge on workspaces is faithfully implemented as action on these circuits, through a coproduct and a Hopf algebra Markov chain. The results obtained here provide a constructive argument showing the theoretical possibility of a neurocomputational realization of the core computational structure of syntax. We also present a particular case of this general construction where this type of realization of Merge is implemented as a cross frequency phase synchronization on sinusoidal waves. This also shows that Merge can be expressed in terms of the successor function of a semiring, thus clarifying the well known observation of its similarities with the successor function of arithmetic.

The syntactic Merge operation of the Minimalist Program in linguistics can be described mathematically in terms of Hopf algebras, with a formalism similar to the one arising in the physics of renormalization. This mathematical formulation of Merge has good descriptive power, as phenomena empirically observed in linguistics can be justified from simple mathematical arguments. It also provides a possible mathematical model for externalization and for the role of syntactic parameters.

In this paper we compare some old formulations of Minimalism, in particular Stabler's computational minimalism, and Chomsky's new formulation of Merge and Minimalism, from the point of view of their mathematical description in terms of Hopf algebras. We show that the newer formulation has a clear advantage purely in terms of the underlying mathematical structure. More precisely, in the case of Stabler's computational minimalism, External Merge can be described in terms of a partially defined operated algebra with binary operation, while Internal Merge determines a system of right-ideal coideals of the Loday-Ronco Hopf algebra and corresponding right-module coalgebra quotients. This mathematical structure shows that Internal and External Merge have significantly different roles in the old formulations of Minimalism, and they are more difficult to reconcile as facets of a single algebraic operation, as desirable linguistically. On the other hand, we show that the newer formulation of Minimalism naturally carries a Hopf algebra structure where Internal and External Merge directly arise from the same operation. We also compare, at the level of algebraic properties, the externalization model of the new Minimalism with proposals for assignments of planar embeddings based on heads of trees.

We extend our formulation of Merge and Minimalism in terms of Hopf algebras to an algebraic model of a syntactic-semantic interface. We show that methods adopted in the formulation of renormalization (extraction of meaningful physical values) in theoretical physics are relevant to describe the extraction of meaning from syntactic expressions. We show how this formulation relates to computational models of semantics and we answer some recent controversies about implications for generative linguistics of the current functioning of large language models.

A feedforward neural network using rectified linear units constructs a mapping from inputs to outputs by partitioning its input space into a set of convex regions where points within a region share a single affine transformation. In order to understand how neural networks work, when and why they fail, and how they compare to biological intelligence, we need to understand the organization and formation of these regions. Step one is to design and implement algorithms for exact region enumeration in networks beyond toy examples. In this work, we present parallel algorithms for exact enumeration in deep (and shallow) neural networks. Our work has three main contributions: (1) we present a novel algorithm framework and parallel algorithms for region enumeration; (2) we implement one of our algorithms on a variety of network architectures and experimentally show how the number of regions dictates runtime; and (3) we show, using our algorithm's output, how the dimension of a region's affine transformation impacts further partitioning of the region by deeper layers. To our knowledge, we run our implemented algorithm on networks larger than all of the networks used in the existing region enumeration literature. Further, we experimentally demonstrate the importance of parallelism for region enumeration of any reasonably sized network.

Cognitive computers (κ C ) are intelligent processors advanced from data and information processing to autonomous knowledge learning and intelligence generation. This work presents a retrospective and prospective review of the odyssey toward κ C empowered by transdisciplinary basic research and engineering advances. A wide range of fundamental theories and innovative technologies for κ C is explored, and a set of underpinning intelligent mathematics (IM) is created. The architectures of κ C for cognitive computing and Autonomous Intelligence Generation (AIG) are designed as a brain-inspired cognitive engine. Applications of κ C in autonomous AI (AAI) are demonstrated by pilot projects. This work reveals that AIG will no longer be a privilege restricted only to humans via the odyssey to κ C toward training-free and self-inferencing computers.

The goal of this contribution to the Elements series is to closely examine Merge, its form, its function, and its central role in current linguistic theory. It explores what it does (and does not do), why it has the form it has, and its development over time. The basic idea behind Merge is quite simple. However, Merge interacts, in intricate ways, with other components including the language's interfaces, laws of nature, and certain language-specific conditions. Because of this, and because of its fundamental place in the human faculty of language, this Element's focus on Merge provides insights into the goals and development of generative grammar more generally, and its prospects for the future.

Using Phylogenetic Algebraic Geometry, we analyze computationally the phylogenetic tree of subfamilies of the Indo-European language family, using data of syntactic structures. The two main sources of syntactic data are the SSWL database and Longobardi’s recent data of syntactic parameters. We compute phylogenetic invariants and estimates of the Euclidean distance functions for two sets of Germanic languages, a set of Romance languages, a set of Slavic languages and a set of early Indo-European languages, and we compare the results with what is known through historical linguistics.

A key goal of cognitive computing is to correctly model human language. Recently, much has been made of the ability of deep neural nets trained on huge datasets to precisely parse sentences. But do these systems truly incorporate human knowledge of language? In this paper we apply a standard linguistic methodology, transformational analysis, to determine whether this claim is accurate. On this view, if a deep net parser operates properly on one kind of sentence, it should also work correctly on its transformed counterpart. Applying this to a standard set of statement-question transformed sentence pairs, we find that a state of the art neural network system does not replicate human behavior and makes numerous errors. We suggest that this kind of test is more relevant for highlighting what deep neural networks can and cannot do with respect to human language.

Cognitive Informatics (CI) and Cognitive Computing (CC) are fundamental intelligence theories and general AI technologies triggered by the transdisciplinary advances in intelligence, computer, brain, knowledge, cognitive, robotic, and cybernetic sciences for engineering implementations. This paper presents a summary of the plenary panel (Part I) on the theoretical foundations of CI/CC as well recent breakthroughs in AI engineering reported in the 20th IEEE International ICCI*CC Conference (ICCI*CC'21). The latest advances in CI and CC towards general AI are presented by twenty-two distinguished panelists. Strategic AI engineering applications in CI, CC, and cognitive systems are elaborated for abstract intelligence, general AI, cognitive robots, autonomous systems, intelligent vehicles, and safety-and-mission-critical systems.

Accurate recovery of predicate-argument structure from a Universal Dependency (UD) parse is central to downstream tasks such as extraction of semantic roles or event representations. This study introduces compchains, a categorization of the hierarchy of predicate dependency relations present within a UD parse. Accuracy of compchain classification serves as a proxy for measuring accurate recovery of predicate-argument structure from sentences with embedding. We analyzed the distribution of compchains in three UD English treebanks, EWT, GUM and LinES, revealing that these treebanks are sparse with respect to sentences with predicate-argument structure that includes predicate-argument embedding. We evaluated the CoNLL 2018 Shared Task UDPipe (v1.2) baseline (dependency parsing) models as compchain classifiers for the EWT, GUMS and LinES UD treebanks. Our results indicate that these three baseline models exhibit poorer performance on sentences with predicate-argument structure with more than one level of embedding; we used compchains to characterize the errors made by these parsers and present examples of erroneous parses produced by the parser that were identified using compchains. We also analyzed the distribution of compchains in 58 non-English UD treebanks and then used compchains to evaluate the CoNLL’18 Shared Task baseline model for each of these treebanks. Our analysis shows that performance with respect to compchain classification is only weakly correlated with the official evaluation metrics (LAS, MLAS and BLEX). We identify gaps in the distribution of compchains in several of the UD treebanks, thus providing a roadmap for how these treebanks may be supplemented. We conclude by discussing how compchains provide a new perspective on the sparsity of training data for UD parsers, as well as the accuracy of the resulting UD parses.

Current deep neural networks have made remarkable advances in their ability to analyze and use natural language, with great apparent engineering success. But how well do these systems mirror the cognitive constraints associated with human language? In this talk we show that there are three essential core computations that characterize human language as an engine of human thought. One is "digital infinity"– the fact that we can produce an open-ended countably infinite number of sentences. The second is that sentences are hierarchically structured, rather than being arranged in a linear array. The third property is that human language computations always admit the possibility of "displacement" – a word or phrase can be pronounced at a place distinct from its usual location of semantic interpretation. All three properties can be shown to follow from a single, simple, recursive combinatorial operation. We provide empirical evidence for all three properties, both from concrete developmental examples as well as psycholinguistic and brain imaging experiments.What about current "deep neural network" systems? Although they perform very well after large-scale training, their success appears to be grounded on accurate table-lookup–memorization–without truly mirroring the three key computational principles of human language cognition. By "stress testing" currently available deep neural network processors, we show that they are, perhaps surprisingly very fragile when presented even with simple examples that deviate modestly from the examples on which they were trained. In particular, they fail to properly represent hierarchical structure and they cannot reliably reconstruct examples of sentences with "displacement" if the examples go just a bit beyond the complexity of their training set data. For example, while a deep neural network system might work on "Which cookie did Bob want," it fails on, "Which cookie did Bob want to eat." Such failures indicate that the neural net systems have not generalized in the same sense that children do, since children can easily handle such examples after receiving much more limited training data.

We consider two different data sets of syntactic parameters and we discuss how to detect relations between parameters through a heat kernel method developed by Belkin-Niyogi, which produces low dimensional representations of the data, based on Laplace eigenfunctions, that preserve neighborhood information. We analyze the different connectivity and clustering structures that arise in the two datasets, and the regions of maximal variance in the two-parameter space of the Belkin-Niyogi construction, which identify preferable choices of independent variables. We compute clustering coefficients and their variance.

In their Essay on the evolution of human language, Martins and Boeckx seek to refute what they call the "half-Merge fallacy"-the conclusion that the most elementary computational operation for human language syntax, binary set formation, or "Merge," evolved in a single step. We show that their argument collapses. It is based on a serious misunderstanding of binary set formation as well as formal language theory. Furthermore, their specific evolutionary scenario counterproposal for a "two-step" evolution of Merge does not work. Although we agree with their Essay on several points, including that there must have been many steps in the evolution of human language and the importance of understanding how language and language syntax are implemented in the brain, we disagree that there is any justification, empirical or conceptual, for the decomposition of binary set formation into separate steps.

Studies of parsing inspired by the Minimalist Program have various goals. They flesh out and unify proposals in syntax; they define properties of fundamental structure-building mechanisms; and they provide mechanisms for psycholinguistic models. In this brief overview, some historical antecedents are noted, and then foundational perspectives underpinning some recent work in this tradition are outlined, with associated basic results on locality and efficiency. Finally, a comparative guide to some notational differences is presented.

Recent neural network architectures such as the basic recurrent neural network (RNN) and Gated Recurrent Unit (GRU) have gained prominence as end-to-end learning architectures for natural language processing tasks. But what is the computational power of such systems? We prove that finite precision RNNs with one hidden layer and ReLU activation and finite precision GRUs are exactly as computationally powerful as deterministic finite automata. Allowing arbitrary precision, we prove that RNNs with one hidden layer and ReLU activation are at least as computationally powerful as pushdown automata. If we also allow infinite precision, infinite edge weights, and nonlinear output activation functions, we prove that GRUs are at least as computationally powerful as pushdown automata. All results are shown constructively.

In a response to Cedric Boeckx, Robert Berwick and Noam Chomsky defend and update their argument that the human language faculty is a species-specific property, with no known group differences and little variation.

While long short-term memory (LSTM) neural net architectures are designed to capture sequence information, human language is generally composed of hierarchical structures. This raises the question as to whether LSTMs can learn hierarchical structures. We explore this question with a well-formed bracket prediction task using two types of brackets modeled by an LSTM. Demonstrating that such a system is learnable by an LSTM is the first step in demonstrating that the entire class of CFLs is also learnable. We observe that the model requires exponential memory in terms of the number of characters and embedded depth, where a sub-linear memory should suffice. Still, the model does more than memorize the training input. It learns how to distinguish between relevant and irrelevant information. On the other hand, we also observe that the model does not generalize well. We conclude that LSTMs do not learn the relevant underlying context-free rules, suggesting the good overall performance is attained rather by an efficient way of evaluating nuisance variables. LSTMs are a way to quickly reach good results for many natural language tasks, but to understand and generate natural language one has to investigate other concepts that can make more direct use of natural language’s structural nature.

The faculty of language is thought to be uniquely human. Recently, it has been claimed that songbirds are able to associate meaning with sound, comparable to the way that humans do. In human language, the meaning of expressions (semantics) is dependent on a mind-internal hierarchical structure (syntax). Meaning is associated with structure through the principle of compositionality, whereby the meaning of a complex expression is a function of the meaning of its constituent parts and the mode of composition. We argue that while recent experimental findings on songbird call sequences offer exciting novel insights into animal communication, despite claims to the contrary, they are quite unlike what we find in human language. There are indeed remarkable behavioral and neural parallels in auditory-vocal imitation learning between songbirds and human infants that are absent in our closest evolutionary relatives, the great apes. But so far, there is no convincing evidence of syntax-determined meaning in nonhuman animals.

In this Formal Comment the authors respond to objections to their previous Essay, reiterating that comparative linguistics is not an easy undertaking.