Generative AI has a problem with trust. Knowledge-based AI does not, but faces the knowledge bottleneck: the cost of manually building the lexicons and ontologies that enable reasoning, explanation, and targeted correction. We present a neurosymbolic approach toward overcoming this bottleneck. This approach uses a symbolic semantic language interpreter and static knowledge resources (ontology, lexicon, and a corpus of 1,780 validated text meaning representations) of OntoAgent, a cognitive architecture, in coordination with a language model guided by narrative descriptions of decision-making algorithms that encode principles from decades of knowledge acquisition research. The acquisition process: a) is triggered when the semantic anlyzer encounters knowledge gaps while constructing meanng representations; b) proposes new lexicon entries and ontological concepts; and c) automatically validates them against ontological constraints. We report the results of an initial experiment using this approach as the first step toward establishing what an LLM can achieve when boot-strapped from deep knowledge resources and integrated with a knowledge-based analyzer. This work describes a step in an ongoing R&D program whose core objective is to demonstrate that, just as GPUs enabled neural methods to scale, the use of language models as knowledge acquisition tools will enable knowledge-based AI to scale.

Deploying embodied AI agents in the physical world demands cognitive capabilities for long-horizon planning that execute reliably, deterministically, and transparently. We present HARMONIC, a cognitive-robotic architecture that pairs OntoAgent, a content-centric cognitive architecture providing metacognitive self-monitoring, domain-grounded diagnosis, and consequence-based action selection over ontologically structured knowledge, with a modular reactive tactical layer. HARMONIC's modular design enables a functional evaluation of whether LLMs can replicate OntoAgent's cognitive capabilities, evaluated within the same robotic system under identical conditions. Six LLMs spanning frontier and efficient tiers replace OntoAgent in a collaborative maintenance scenario under native and knowledge-equalized conditions. Results reveal that LLMs do not consistently assess their own knowledge state before acting, causing downstream failures in diagnostic reasoning and action selection. These deficits persist even with equivalent procedural knowledge, indicating the issues are architectural rather than knowledge-based. These findings support the design of physically embodied systems in which cognitive architectures retain primary authority for reasoning, owing to their deterministic and transparent characteristics.

Robots operating alongside humans must recognize what they do not know before acting, diagnose problems from domain knowledge, and reason about action consequences. These capabilities are operational requirements, not optimization targets, and their absence produces silent and unrecoverable failures. We present a first-of-its-kind controlled comparison between OntoAgent, our content-centric cognitive architecture, and six LLMs spanning frontier and efficient tiers as drop-in replacements at the strategic layer of the same robotic system in HARMONIC. LLMs fail to verify their knowledge state before acting, even when given equivalent procedural knowledge. The deficit is architectural, not knowledge-based. Knowledge-grounded architectures must retain decision authority; LLMs contribute where their strengths apply.

Multiagent and multirobot systems (MRS) often rely on direct communication for information sharing. This work explores an alternative approach inspired by eavesdropping mechanisms in nature that involves casual observation of agent interactions to enhance decentralized knowledge dissemination. We achieve this through a novel indirect knowledge transfer through behavior trees (IKT-BT) framework tailored for a behavior-based MRS, encapsulating knowledge and control actions in behavior trees (BT). We present two new BT-based modalities—eavesdrop-update (EU) and eavesdrop-buffer-update (EBU)—incorporating unique eavesdropping strategies and efficient episodic memory management suited for resource-limited swarm robots. We theoretically analyze the IKT-BT framework for an MRS and validate the performance of the proposed modalities through extensive experiments simulating a search and rescue mission. Our results reveal improvements in both global mission performance outcomes and agent-level knowledge dissemination with a reduced need for direct communication.

Shapes of cognition is a new conceptual paradigm for the computational cognitive modeling of Language-Endowed Intelligent Agents (LEIAs). Shapes are remembered constellations of sensory, linguistic, conceptual, episodic, and procedural knowledge that allow agents to cut through the complexity of real life the same way as people do: by expecting things to be typical, recognizing patterns, acting by habit, reasoning by analogy, satisficing, and generally minimizing cognitive load to the degree situations permit. Atypical outcomes are treated using shapes-based recovery methods, such as learning on the fly, asking a human partner for help, or seeking an actionable, even if imperfect, situational understanding. Although shapes is an umbrella term, it is not vague: shapes-based modeling involves particular objectives, hypotheses, modeling strategies, knowledge bases, and actual models of wide-ranging phenomena, all implemented within a particular cognitive architecture. Such specificity is needed both to vet our hypotheses and to achieve our practical aims of building useful agent systems that are explainable, extensible, and worthy of our trust, even in critical domains. However, although the LEIA example of shapes-based modeling is specific, the principles can be applied more broadly, giving new life to knowledge-based and hybrid AI.

We report a neurosymbolic approach to authorship anonymization that combines knowledge-based paraphrasing, grounded in cognitive modeling, with support functions provided by a large language model (LLM). The cognitive model accounts for four things: what it means to faithfully retain meaning and discourse coherence in a paraphrase, how do deal with polysemy given that full semantic analysis of open text is beyond the state of the art, how to define and characterize an author’s style, and how to leverage human linguistic capabilities when preparing systems to automatically anonymize texts. LLMs augment the knowledge-based paraphrases in three ways: by filtering out atypical formulations, by selecting the best from multiple candidate paraphrases, and by offering additional paraphrases in case the knowledge-based paraphrasing fails to adequately anonymize the text. This neurosymbolic architecture favors knowledge-based processing for being reliable and explainable, while exploiting LLMs for what they do best: manipulate regularities in the surface form of language.

This paper introduces HARMONIC, a cognitive-robotic architecture designed for robots in human-robotic teams. HARMONIC supports semantic perception interpretation, human-like decision-making, and intentional language communication. It addresses the issues of safety and quality of results; aims to solve problems of data scarcity, explainability, and safety; and promotes transparency and trust. Two proof-of-concept HARMONIC-based robotic systems are demonstrated, each implemented in both a high-fidelity simulation environment and on physical robotic platforms.

For AI agents to emulate human behavior, they must be able to perceive, meaningfully interpret, store, and use large amounts of information about the world, themselves, and other agents. Metacognition is a necessary component of all of these processes. In this paper, we briefly a) introduce content-centric computational cognitive (C4) modeling for next-generation AI agents; b) review the long history of developing C4 agents at RPI's LEIA (Language-Endowed Intelligent Agents) Lab; c) discuss our current work on extending LEIAs' cognitive capabilities to cognitive robotic applications developed using a neuro symbolic processing model; and d) sketch plans for future developments in this paradigm that aim to overcome underappreciated limitations of currently popular, LLM-driven methods in AI.

Explanation is key to people having confidence in high-stakes AI systems. However, machine-learning-based systems - which account for almost all current AI - can't explain because they are usually black boxes. The explainable AI (XAI) movement hedges this problem by redefining "explanation". The human-centered explainable AI (HCXAI) movement identifies the explanation-oriented needs of users but can't fulfill them because of its commitment to machine learning. In order to achieve the kinds of explanations needed by real people operating in critical domains, we must rethink how to approach AI. We describe a hybrid approach to developing cognitive agents that uses a knowledge-based infrastructure supplemented by data obtained through machine learning when applicable. These agents will serve as assistants to humans who will bear ultimate responsibility for the decisions and actions of the human-robot team. We illustrate the explanatory potential of such agents using the under-the-hood panels of a demonstration system in which a team of simulated robots collaborates on a search task assigned by a human.

Multirobot and multiagent systems demonstrate collective (swarm) intelligence through systematic and distributed integration of local behaviors in a group. Agents sharing knowledge about the mission and environment can enhance performance at individual and mission levels. However, this is difficult to achieve, partly due to the lack of a generic framework for transferring part of the known knowledge (behaviors) between agents. This article presents a new knowledge representation framework and a transfer strategy called KT-BT: knowledge transfer through behavior trees. The KT-BT framework follows a query-response-update mechanism through an online behavior tree framework, where agents broadcast queries for unknown conditions and respond with appropriate knowledge using a condition-action-control subflow. We embed a novel grammar structure called stringBT that encodes knowledge, enabling behavior sharing. We theoretically investigate the properties of the KT-BT framework in achieving homogeneity of high knowledge across the entire group compared to a heterogeneous system without the capability of sharing their knowledge. We extensively verify our framework in a simulated multirobot search and rescue problem. The results show successful knowledge transfers and improved group performance in various scenarios. We further study the effects of opportunities and communication range on group performance, knowledge spread, and functional heterogeneity in a group of agents, presenting interesting insights.

We present HARMONIC, a framework for implementing cognitive robots that transforms general-purpose robots into trusted teammates capable of complex decision-making, natural communication and human-level explanation. The framework supports interoperability between a strategic (cognitive) layer for high-level decision-making and a tactical (robot) layer for low-level control and execution. We describe the core features of the framework and our initial implementation, in which HARMONIC was deployed on a simulated UGV and drone involved in a multi-robot search and retrieval task.

This paper describes HARMONIC, a cognitive-robotic architecture that integrates the OntoAgent cognitive framework with general-purpose robot control systems applied to human-robot teaming (HRT). HARMONIC incorporates metacognition, meaningful natural language communication, and explainability capabilities required for developing mutual trust in HRT. Through simulation experiments involving a joint search task performed by a heterogeneous team of two HARMONIC-based robots and a human operator, we demonstrate heterogeneous robots that coordinate their actions, adapt to complex scenarios, and engage in natural human-robot communication. Evaluation results show that HARMONIC-based robots can reason about plans, goals, and team member attitudes while providing clear explanations for their decisions, which are essential requirements for realistic human-robot teaming.

The realism and immersion of Virtual Reality (VR) experiences depend on the quality of interactions between the virtual hand and virtual objects. The current drawbacks, such as seemingly artificial hand-object interactions and erratic post-collision behaviors of both virtual objects and the virtual hand, curtail the effectiveness of VR in tasks requiring precise manipulation, precluding sustained and successful adoption of VR in precision manual tasks. To address these limitations, we advocate a strategic approach across three related domains: (1) Developing a sufficiently realistic virtual hand model (i.e., a set of rigid bodies or deformable meshes) that can implement the complex movements of a biological hand. (2) Exploiting synergistic patterns of multi-digit motion and contact forces revealed by research on neuroscience, psychophysics, and manual actions to develop hand-object collision handling algorithms. (3) Implementing seamless and fluid releases of whole-hand virtual grasps, especially involving complex grasps and in-hand manipulation tasks. This article explores various aspects of virtual grasping that go beyond traditional physics simulations, proposing innovative solutions to overcome technical barriers across these domains.

Experiments using large numbers of miniature swarm robots are desirable to teach, study, and test multi-robot and swarm intelligence algorithms and their applications. To realize the full potential of a swarm robot, it should be capable of not only motion but also sensing, computing, communication, and power management modules with multiple options. Current swarm robot platforms developed for commercial and academic research purposes lack several of these critical attributes by focusing only on a few of these aspects. Therefore, in this paper, we propose the HeRoSwarm, a fully-capable swarm robot platform with open-source hardware and software support. The proposed robot hardware is a low-cost design with commercial off-the-shelf components that uniquely integrates multiple sensing, communication, and computing modalities with various power management capabilities into a tiny footprint. Moreover, our swarm robot with odometry capability with Robot Operating Systems (ROS) support is unique in its kind. This simple yet powerful swarm robot design has been extensively verified with different prototyping variants and multi-robot experimental demonstrations.

The paper describes a system that uses large language model (LLM) technology to support the automatic learning of new entries in an intelligent agent's semantic lexicon. The process is bootstrapped by an existing non-toy lexicon and a natural language generator that converts formal, ontologically-grounded representations of meaning into natural language sentences. The learning method involves a sequence of LLM requests and includes an automatic quality control step. To date, this learning method has been applied to learning multiword expressions whose meanings are equivalent to those of transitive verbs in the agent's lexicon. The experiment demonstrates the benefits of a hybrid learning architecture that integrates knowledge-based methods and resources with both traditional data analytics and LLMs.

Design of complex self-assembly systems requires intelligent solutions that can be manufactured effectively and efficiently. Self-organization is the spontaneous formation of organized structures that can dynamically reconfigure with changing environments. These processes are primarily observed in chemical and biological processes that resemble large-scale ecosystems and in environments as small as biological cells. Inspired by these natural processes, there is also a growing research interest in developing 4D design and printing technologies in which 3D structures reconfigure with changing stimuli. The 4D design process requires appropriate design, computational, and simulation tools aimed at building structures at larger scales that can augment the current engineering design and manufacturing processes. This study presents a new multi-agent framework with two new paradigms called agents-as-blocks and free-agent. We present further details on these new strategies in the form of preliminary case studies applied to simulating micro-environments of microtubules’ self-organization process and through a vibration simulation platform. Our simulation results closely follow the real formation patterns in the microtubules process and show some interesting self-organizing and self-assembling patterns that change with varying geometries, rules, and stimuli in a vibration platform environment.

Achieving knowledge sharing within an artificial swarm system could lead to significant development in autonomous multiagent and robotic systems research and realize collective intelligence. However, this is difficult to achieve since there is no generic framework to transfer skills between agents other than a query-response-based approach. Moreover, natural living systems have a "forgetfulness" property for everything they learn. Analyzing such ephemeral nature (temporal memory properties of new knowledge gained) in artificial systems has never been studied in the literature. We propose a behavior tree-based framework to realize a query-response mechanism for transferring skills encoded as the condition-action control sub-flow of that portion of the knowledge between agents to fill this gap. We simulate a multiagent group with different initial knowledge on a foraging mission. While performing basic operations, each robot queries other robots to respond to an unknown condition. The responding robot shares the control actions by sharing a portion of the behavior tree that addresses the queries. Specifically, we investigate the ephemeral nature of the new knowledge gained through such a framework, where the knowledge gained by the agent is either limited due to memory or is forgotten over time. Our investigations show that knowledge grows proportionally with the duration of remembrance, which is trivial. However, we found minimal impact on knowledge growth due to memory. We compare these cases against a baseline that involved full knowledge pre-coded on all agents. We found that knowledge-sharing strived to match the baseline condition by sharing and achieving knowledge growth as a collective system.

Patients whose lungs are compromised due to various respiratory health concerns require mechanical ventilation for support in breathing. Different mechanical ventilation settings are selected depending on the patient's lung condition, and the selection of these parameters depends on the observed patient response and experience of the clinicians involved. To support this decision-making process for clinicians, good prediction models are always beneficial in improving the setting accuracy, reducing treatment error, and quickly weaning patients off the ventilation support. In this study, we developed a machine learning model for estimation of the mechanical ventilation parameters for lung health. The model is based on inverse mapping of artificial neural networks with the Graded Particle Swarm Optimizer. In this new variant, we introduced grouping and hierarchy in the swarm in addition to the general rules of particle swarm optimization to further improve its prediction performance of the mechanical ventilation parameters. The machine learning model was trained and tested using clinical data from canine and feline patients at the University of Georgia College of Veterinary Medicine. Our model successfully generated a range of parameter values for the mechanical ventilation applied on test data, with the average prediction values over multiple trials close to the target values. Overall, the developed machine learning model should be able to predict the mechanical ventilation settings for various respiratory conditions for patient's survival once the relevant data are available.

Many species in nature demonstrate symbiotic relationships leading to emergent behaviors through cooperation, which are sometimes beyond the scope of the partnerships within the same species. These symbiotic relationships are classified as mutualism, commensalism, and parasitism based on the benefit levels involved. While these partnerships are ubiquitous in nature, it is imperative to understand the benefits of collective behaviors in designing heterogeneous multi-robot systems (HMRS). In this paper, we investigate the impact of heterogeneity on the performance of HMRS applied to a search and rescue problem. The groups consisting of searchers and rescuers, varied in the individual robot behaviors with multiple degrees of functionality overlap and group compositions, demonstrating various levels of heterogeneity. We propose a new technique to measure heterogeneity in the agents through the use of Behavior Trees and use it to obtain heterogeneity informatics from our Monte Carlo simulations. The results show a positive correlation between the groups' heterogeneity measure and the rescue efficiency demonstrating benefits in most of the scenarios. However, we also see cases where heterogeneity may hamper the group's abilities pointing to the need for determining the optimal heterogeneity in group required to maximally benefit from HMRS in real-world applications.

Microtubules (MTs) are highly dynamic polymers distributed in the cytoplasm of a biological cell. Alpha and beta globular proteins constituting the heterodimer building blocks combine to form these tubules through polymerization, controlled by the concentration of Guanosine-triphosphate (GTPs) and other Microtubule Associated Proteins (MAPs). MTs play a crucial role in many intracellular processes, predominantly in mitosis, organelle transport and cell locomotion. Current research in this area is focused on understanding the exclusive behaviors of self-organization and their association with different MAPs through organized laboratory experiments. However, the intriguing intelligence behind these tiny machines resulting in complex self-organizing structures is mostly unexplored. In this study, we propose a novel swarm engineering framework in modeling rules for these systems, by combining the principles of design with swarm intelligence. The proposed framework was simulated on a game engine and these simulations demonstrated self-organization of rings and protofilaments in MTs. Analytics from these simulations assisted in understanding the influence of GTPs on protofilament formation. Also, results showed that the population density of GTPs rather than their bonding probabilities played a crucial role in polymerization in forming microtubule substructures.