Abstract General intelligence must reorganize experience into internal structures that enable prediction and action under finite resources. Existing systems implicitly presuppose fixed primitive units— tokens, subwords, pixels, or predefined sensor channels—thereby bypassing the question of how representational units themselves emerge and stabilize. This paper proposes Self-organizing Active Network of Concepts with Energy E 3 – SANC( E 3), an axiomatic framework in which representational units are not given a priori but instead arise as stable outcomes of competitive selection, reconstruction, and compression under finite activation capacity, governed by the explicit minimization of an energy functional E 3 = λ 1 L rec + λ 2 C struct + λ 3 C update . SANC( E 3 ) draws a principled distinction between system tokens —structural anchors such as {here, now, I} and sensory sources—and tokens that emerge through self-organization during co-occurring events. Five core axioms formalize finite capacity, association from co-occurrence, similarity-based competition, confidence-based stabilization, and the reconstruction–compression– update trade-off. Residual capacity C remain ( t ) is introduced as an explicit control variable that dynamically modulates representation creation, stabilization, and deletion thresholds. A key feature of the framework is a pseudo-memory-mapped I/O mechanism , through which internally replayed Gestalts are processed via the same axiomatic pathway as external sensory input. At the theoretical level, perception, imagination, prediction, planning, and action are unified within a single representational and energetic process. This unification is intended as a structural account: it does not claim that a complete implemented AGI system is provided here, but specifies how these processes can be described within one axiomatic pathway. The same account naturally extends the framework to embodied and physical agents, where interaction with the environment and motor behavior are treated as continuations of Gestalt completion rather than as separate control modules. From the axioms, twelve propositions are derived, suggesting how category formation, automatic threshold tuning, hierarchical organization, unsupervised learning, and high-level cognitive activities—dialogue, authoring, perception, causal inference, and action—can be understood as instances of Gestalt completion under E 3 minimization .
Looking back on my intellectual career of nearly a half century, from the 1970s to the 2020s, it is surprising— at least to me—that it was not until I began this memoir that I understood it was all about architectural exploration . Initially this focused on cognitive architectures for integrated minds , such as Soar and Sigma, but increasingly it has also involved architectures of academic disciplines —such as artificial (general) intelligence and computer/cognitive science—that study integrated minds. The theme of this memoir then emerged naturally from understanding this methodology—including a particular perspective on what it means to be an architecture and the continuing importance of exploration—and of its application to this pair of apparently disparate topics. This memoir thus develops this theme and traces it through my career.
Abstract Shared conceptualization, in the sense we take it here, is as recent a notion as the Semantic Web, but its relevance for a large variety of fields requires efficient methods of extraction and representation for both quantitative and qualitative data. This notion is particularly relevant for the investigation into, and construction of, semantic structures such as knowledge bases and taxonomies, but given the required large, often inaccurate, corpora available for search we can get only approximations. We see fuzzy description logic as an adequate medium for the representation of human semantic knowledge and propose a means to couple it with fuzzy semantic networks via the propositional Łukasiewicz fuzzy logic such that these suffice for decidability for queries over a semantic-knowledge base such as “to what degree of sharedness does it entail the instantiation C(a) for some concept C” or “what are the roles R that connect the individuals a and b to degree of sharedness ε.”
AbstractThere has been considerable recent interest in the “The Reward is Enough” hypothesis, which is the idea that agents can develop general intelligence even with simple reward functions, provided the environment they operate in is sufficiently complex. While this is an interesting framework to approach the AGI problem, it also brings forth new questions - what kind of RL algorithm should the agent use? What should the reward function look like? How can it quickly generalize its learning to new tasks? This paper looks to animal reinforcement learning - both individual and social - to address these questions and more. It evaluates existing computational models and neural substrates of Pavlovian conditioning, reward-based action selection, intrinsic motivation, attention-based task representations, social learning and meta-learning in animals and discusses how insights from these findings can influence the development of animal-level AGI within an RL framework.
We consider an extended notion of reinforcement learning in which the environment can simulate the agent and base its outputs on the agent’s hypothetical behavior. Since good performance usually requires paying attention to whatever things the environment’s outputs are based on, we argue that for an agent to achieve on-average good performance across many such extended environments, it is necessary for the agent to self-reflect. Thus weighted-average performance over the space of all suitably well-behaved extended environments could be considered a way of measuring how self-reflective an agent is. We give examples of extended environments and introduce a simple transformation which experimentally seems to increase some standard RL agents’ performance in a certain type of extended environment.
Abstract On the basis of the author’s earlier works, the article proposes a new approach to creating an artificial intellect system in a model of a human being that is presented as the unification of an intellectual agent and a humanoid robot (ARb). In accordance with the proposed new approach, the development of an artificial intellect is achieved by teaching a natural language to an ARb, and by its utilization for communication with ARbs and humans, as well as for reflections. A method is proposed for the implementation of the approach. Within the framework of that method, a human model is “brought up” like a child, in a collective of automatons and children, whereupon an ARb must master a natural language and reflection, and possess self-awareness. Agent robots (ARbs) propagate and their population evolves; that is ARbs develop cognitively from generation to generation. ARbs must perform the tasks they were given, such as computing, whereupon they are then assigned time for “private life” for improving their education as well as for searching for partners for propagation. After having received an education, every agent robot may be viewed as a “person” who is capable of activities that contain elements of creativity. The development of ARbs thanks to the evolution of their population, education, and personal “life” experience, including “work” experience, which is mastered in a collective of humans and automatons.
Abstract The Feature Markov Decision Processes ( MDPs) model developed in Part I (Hutter, 2009b) is well-suited for learning agents in general environments. Nevertheless, unstructured (Φ)MDPs are limited to relatively simple environments. Structured MDPs like Dynamic Bayesian Networks (DBNs) are used for large-scale real-world problems. In this article I extend ΦMDP to ΦDBN. The primary contribution is to derive a cost criterion that allows to automatically extract the most relevant features from the environment, leading to the “best” DBN representation. I discuss all building blocks required for a complete general learning algorithm, and compare the novel ΦDBN model to the prevalent POMDP approach.
Abstract Thinking machines must be able to use language effectively in communication with humans. It requires from them the ability to generate meaning and transfer this meaning to a communicating partner. Machines must also be able to decode meaning communicated via language. This work is about meaning in the context of building an artificial general intelligent system. It starts with an analysis of the Turing test and some of the main approaches to explain meaning. It then considers the generation of meaning in the human mind and argues that meaning has a dual nature. The quantum component reflects the relationships between objects and the orthogonal quale component the value of these relationships to the self. Both components are necessary, simultaneously, for meaning to exist. This parallel existence permits the formulation of ‘meaning coordinates’ as ordered pairs of quantum and quale strengths. Meaning coordinates represent the contents of meaningful mental states. Spurred by a currently salient meaningful mental state in the speaker, language is used to induce a meaningful mental state in the hearer. Therefore, thinking machines must be able to produce and respond to meaningful mental states in ways similar to their functioning in humans. It is explained how quanta and qualia arise, how they generate meaningful mental states, how these states propagate to produce thought, how they are communicated and interpreted, and how they can be simulated to create thinking machines.
In 2011, Hibbard suggested an intelligence measure for agents who compete in an adversarial sequence prediction game. We argue that Hibbard's idea should actually be considered as two separate ideas: first, that the intelligence of such agents can be measured based on the growth rates of the runtimes of the competitors that they defeat; and second, one specific (somewhat arbitrary) method for measuring said growth rates. Whereas Hibbard's intelligence measure is based on the latter growth-rate-measuring method, we survey other methods for measuring function growth rates, and exhibit the resulting Hibbard-like intelligence measures and taxonomies. Of particular interest, we obtain intelligence taxonomies based on Big-O and Big-Theta notation systems, which taxonomies are novel in that they challenge conventional notions of what an intelligence measure should look like. We discuss how intelligence measurement of sequence predictors can indirectly serve as intelligence measurement for agents with Artificial General Intelligence (AGIs).
All commentaries were accepted after peer-review.If the reader was expecting a consensus around defining AI, we are afraid we have to disappoint them.We have received many kinds of responses: commentators that don't agree with Pei Wang's definition and provide their own, those that don't consider we need new definitions at all, those that agree with Wang's but still provide a new definition of AI, as well as those that additionally prefer to comment about other topics they feel are also important.A very colored spectrum around defining the most important concept of the AI field!
AbstractAfter generalizing the Archimedean property of real numbers in such a way as to make it adaptable to non-numeric structures, we demonstrate that the real numbers cannot be used to accurately measure non-Archimedean structures. We argue that, since an agent with Artificial General Intelligence (AGI) should have no problem engaging in tasks that inherently involve non-Archimedean rewards, and since traditional reinforcement learning rewards are real numbers, therefore traditional reinforcement learning probably will not lead to AGI. We indicate two possible ways traditional reinforcement learning could be altered to remove this roadblock.
Abstract Natural selection has imbued biological agents with motivations moving them to act for survival and reproduction, as well as to learn so as to support both. Artificial agents also require motivations to act in a goal-directed manner and to learn appropriately into various memories. Here we present a biologically inspired motivation system, based on feelings (including emotions) integrated within the LIDA cognitive architecture at a fundamental level. This motivational system, operating within LIDA’s cognitive cycle, provides a repertoire of motivational capacities operating over a range of time scales of increasing complexity. These include alarms, appraisal mechanisms, appetence and aversion, and deliberation and planning.
Abstract Although animals such as spiders, fish, and birds have very different anatomies, the basic mechanisms that govern their perception, decision-making, learning, reproduction, and death have striking similarities. These mechanisms have apparently allowed the development of general intelligence in nature. This led us to the idea of approaching artificial general intelligence (AGI) by constructing a generic artificial animal (animat) with a configurable body and fixed mechanisms of perception, decision-making, learning, reproduction, and death. One instance of this generic animat could be an artificial spider, another an artificial fish, and a third an artificial bird. The goal of all decision-making in this model is to maintain homeostasis. Thus actions are selected that might promote survival and reproduction to varying degrees. All decision-making is based on knowledge that is stored in network structures. Each animat has two such network structures: a genotype and a phenotype. The genotype models the initial nervous system that is encoded in the genome (“the brain at birth”), while the phenotype represents the nervous system in its present form (“the brain at present”). Initially the phenotype and the genotype coincide, but then the phenotype keeps developing as a result of learning, while the genotype essentially remains unchanged. The model is extended to ecosystems populated by animats that develop continuously according to fixed mechanisms for sexual or asexual reproduction, and death. Several examples of simple ecosystems are given. We show that our generic animat model possesses general intelligence in a primitive form. In fact, it can learn simple forms of locomotion, navigation, foraging, language, and arithmetic.
Legg and Hutter, as well as subsequent authors, considered intelligent agents through the lens of interaction with reward-giving environments, attempting to assign numeric intelligence measures to such agents, with the guiding principle that a more intelligent agent should gain higher rewards from environments in some aggregate sense. In this paper, we consider a related question: rather than measure numeric intelligence of one Legg- Hutter agent, how can we compare the relative intelligence of two Legg-Hutter agents? We propose an elegant answer based on the following insight: we can view Legg-Hutter agents as candidates in an election, whose voters are environments, letting each environment vote (via its rewards) which agent (if either) is more intelligent. This leads to an abstract family of comparators simple enough that we can prove some structural theorems about them. It is an open question whether these structural theorems apply to more practical intelligence measures.
Abstract This article systematically analyzes the problem of defining “artificial intelligence.” It starts by pointing out that a definition influences the path of the research, then establishes four criteria of a good working definition of a notion: being similar to its common usage, drawing a sharp boundary, leading to fruitful research, and as simple as possible. According to these criteria, the representative definitions in the field are analyzed. A new definition is proposed, according to it intelligence means “adaptation with insufficient knowledge and resources.” The implications of this definition are discussed, and it is compared with the other definitions. It is claimed that this definition sheds light on the solution of many existing problems and sets a sound foundation for the field.
This paper presents Unlimited Computable AI, or UCAI, that is a family of computable variants of AIXI. UCAI is more powerful than AIXItl, that is a conventional family of computable variants of AIXI, in the following ways: 1) UCAI supports models of terminating computation, including typed lambda calculus, while AIXItl only supports Turing machine with timeout t, which can be simulated by typed lambda calculus for any t; 2) unlike UCAI, AIXItl limits the program length to l.
Abstract A computational model for artificial animals (animats) interacting with real or artificial ecosystems is presented. All animats use the same mechanisms for learning and decisionmaking. Each animat has its own set of needs and its own memory structure that undergoes continuous development and constitutes the basis for decision-making. The decision-making mechanism aims at keeping the needs of the animat as satisfied as possible for as long as possible. Reward and punishment are defined in terms of changes to the level of need satisfaction. The learning mechanisms are driven by prediction error relating to reward and punishment and are of two kinds: multi-objective local Q-learning and structural learning that alter the architecture of the memory structures by adding and removing nodes. The animat model has the following key properties: (1) autonomy: it operates in a fully automatic fashion, without any need for interaction with human engineers. In particular, it does not depend on human engineers to provide goals, tasks, or seed knowledge. Still, it can operate either with or without human interaction; (2) generality: it uses the same learning and decision-making mechanisms in all environments, e.g. desert environments and forest environments and for all animats, e.g. frog animats and bee animats; and (3) adequacy: it is able to learn basic forms of animal skills such as eating, drinking, locomotion, and navigation. Eight experiments are presented. The results obtained indicate that (i) dynamic memory structures are strictly more powerful than static; (ii) it is possible to use a fixed generic design to model basic cognitive processes of a wide range of animals and environments; and (iii) the animat framework enables a uniform and gradual approach to AGI, by successively taking on more challenging problems in the form of broader and more complex classes of environments
Abstract This paper attempts to replicate the results of evaluating several artificial agents using the Algorithmic Intelligence Quotient test originally reported by Legg and Veness. Three experiments were conducted: One using default settings, one in which the action space was varied and one in which the observation space was varied. While the performance of freq, Q0, Qλ, and HLQλ corresponded well with the original results, the resulting values differed, when using MC-AIXI. Varying the observation space seems to have no qualitative impact on the results as reported, while (contrary to the original results) varying the action space seems to have some impact. An analysis of the impact of modifying parameters of MC-AIXI on its performance in the default settings was carried out with the help of data mining techniques used to identifying highly performing configurations. Overall, the Algorithmic Intelligence Quotient test seems to be reliable, however as a general artificial intelligence evaluation method it has several limits. The test is dependent on the chosen reference machine and also sensitive to changes to its settings. It brings out some differences among agents, however, since they are limited in size, the test setting may not yet be sufficiently complex. A demanding parameter sweep is needed to thoroughly evaluate configurable agents that, together with the test format, further highlights computational requirements of an agent. These and other issues are discussed in the paper along with proposals suggesting how to alleviate them. An implementation of some of the proposals is also demonstrated.
Abstract One of the essential aspect in biological agents is dynamic stability. This aspect, called homeostasis, is widely discussed in ethology, neuroscience and during the early stages of artificial intelligence. Ashby’s homeostats are general-purpose learning machines for stabilizing essential variables of the agent in the face of general environments. However, despite their generality, the original homeostats couldn’t be scaled because they searched their parameters randomly. In this paper, first we re-define the objective of homeostats as the maximization of a multi-step survival probability from the view point of sequential decision theory and probabilistic theory. Then we show that this optimization problem can be treated by using reinforcement learning algorithms with special agent architectures and theoretically-derived intrinsic reward functions. Finally we empirically demonstrate that agents with our architecture automatically learn to survive in a given environment, including environments with visual stimuli. Our survival agents can learn to eat food, avoid poison and stabilize essential variables through theoretically-derived single intrinsic reward formulations.
