Stuart Russell has named “human enfeeblement” as a long-term societal risk of advanced AI. This paper gives a new interpretation of the enfeeblement danger by interpreting it in light of recent work by Brian Christian, Alison Gopnik, and David Spivak on care and caregiving, and connecting it with the ontological design tradition of Terry Winograd and Fernando Flores. The enfeeblement that threatens is not merely a matter of cognitive off-loading and loss of knowledge, but a form of disengagement with the world which I explain in terms of an erosion of our capacities to care. This paper articulates a conception of care as a constellation of cultivable skills that involve us in the world and with each other, while making life worth living. Designing AI to support our capacities to care addresses the worry recently raised by Demis Hassabis and others regarding the questions of “meaning and purpose” that are looming in a post-AGI world.
The defining challenge of AGI research is to build synthetic systems that genuinely instantiate autonomous agency rather than merely producing sophisticated intelligent outputs. While contemporary Large Language Models (LLMs) achieve remarkable reasoning and statistical capacity, they remain structurally fragile and devoid of any intrinsic stake in their own operational survival. This paper addresses that category error with the Agency Spectrum, a multidimensional diagnostic framework that asks not whether a system is an agent but how the competencies constituting agency are organized within it, across diverse substrates. It decouples a system’s capacities along three independent axes rather than scoring it on a single scale. The framework’s theory draws together Active Inference, the Free Energy Principle, and multiscale biology, with a cross-disciplinary synthesis across philosophy, psychology, biology, and computer science. The result is a substrate-agnostic vocabulary: biological and engineered systems can be compared on common terms. We propose a recursive, closed-loop architecture separating the Driver (intrinsic, viability-grounded goal generation), the Engine (generative modeling), and the Interface (the sensorimotor boundary coupling the system to its environment), in which the three components are joined through bidirectional perception–action exchange. Synthesizing Active Inference with principles of multiscale biology, we show why scaling processing throughput cannot generate autonomous, goal-directed behavior. To reach sophisticated agency, architectures must move beyond disembodied computationalism. We identify three necessary architectural conditions—endogenous normativity, recursive substrate-level viability distribution, and structural remapping capacity—and offer four candidate engineering pathways for transitioning from passive information-processing engines to integrated, self-sustaining synthetic agents.
This paper proposes Affective Control under Uncertainty (ACU), an Active Inference architecture in which constitutive viability modulates policy precision. Rather than adding survival or homeostatic state to a reward function, ACU treats internal viability as a control variable that reshapes the urgency, scope, and sharpness of policy competition. An Affective Viability Controller computes a global viability signal V(t) from deviation between current internal state and viable bounds, enabling flexible exploration when viability is secure and adaptive narrowing when viability is threatened. The paper also formalises Self-Model Recruitment Intensity (SMRI), a multiplicative trigger combining policy entropy, variance in expected viability outcomes, and temporal pressure. When thresholded, SMRI recruits a transient self-model with specific consequences: deeper planning, richer interoceptive and environmental integration, computational reallocation, and representation of the agent as a control-relevant object. A minimal survival-gridworld implementation is specified with reward-only and intrinsic-reward baselines. ACU offers a testable route toward uncertainty-aware AGI grounded in constitutive viability rather than externally assigned objectives.
Active Inference has become a leading candidate framework for general intelligence and is increasingly invoked in accounts of consciousness itself. It is often suggested that sufficiently rich inferential organization under the Free Energy Principle will give rise to subjectivity or phenomenal structure. I argue that this step remains theoretically underdetermined: Active Inference provides a principled account of action, perception, and policy selection, but does not by itself specify the structural condition under which a world is given as meaningful for an agent. Drawing on phenomenological analyses of intentional quality, I propose perspective as a minimal structural condition for subjectivity: a slowly evolving, history-dependent orientation that shapes inference and perception without functioning as a variable within the standard inferential economy. I summarize a minimal architecture in which this condition is probed empirically through targeted ablations, and close with implications for AGI, AI alignment, and the ethics of intentionally instantiating perspectival structure.
We present a theory of self-representing cognitive systems grounded in ([0,∞ ],+) -enriched category theory and the Yoneda lemma. The central object is a self-representing ([0,∞ ],+) -enriched category 𝒞 —a Lawvere metric space whose objects are complete epistemic architectures, whose hom-values record directed informational upgrade costs, and which is separated, closed under internal homs, and bilaterally Cauchy complete—together with a contractive cognitive endofunctor F:𝒞→𝒞 modelling iterative self-improvement. We establish eight results in a single logical arc. The Horizon Theorem shows that the Yoneda embedding φ (A)=𝒞(-,A) is never essentially surjective: 𝒞 sits strictly inside its own free Cauchy completion 𝒫(𝒞) , with the non-representable presheaves forming a topologically dense family, proved via a reflexivity argument. The Lawvere–Banach Attractor Theorem shows that every contractive endofunctor on a bilaterally complete, separated ([0,∞ ],+) -enriched category converges to a unique fixed point Ω at a geometric rate. The Boundary Derivation Theorem shows that Ω is the minimal F-invariant substructure of 𝒞 , with all of 𝒞 as its basin of attraction—the constitutional boundary, derived rather than postulated. The Horizon Expansion Theorem shows that each strictly ascending self-modification produces a new, quantitatively distinct non-representable witness. Beyond these four central results, we prove that Kleene and Bourbaki–Witt conditions yield only non-expansiveness when metrised, that contractive endofunctors form a monoid, and that the Yoneda horizon admits an observable diagnostic stabilising in finite time. The architectural section derives structural corrigibility and the alignment-incompleteness duality among five implications. The organising duality is exact: the non-surjectivity of φ and the existence of Ω are two faces of the same ([0,∞ ],+) -enriched structure. Ω inhabits the space between them—not as a postulate, but as a proof. We argue that the eight theorems constitute universal laws of contractive cognitive systems: a stable constitutional boundary is not an engineering design choice but a topological inevitability for any reliably self-improving agent operating within a self-representing enriched metric space. The postulate becomes a theorem. The boundary is not imposed. It emerges.
