The choice of knowledge representation varies among different cognitive architectures. The agent’s experience of Existence is defined by what the knowledge representation can and cannot represent from its observation. In this work, we define a representation of episodes that enable cognitive architectures to accumulate experiences. We describe how Existence may be fragmented in space and time to achieve a core knowledge representation format. This is used to construct Episode Ideas as a form of encoding the agent’s experience of Existence. We demonstrate through experiments the use and versatility of the representation, as well as how semantic knowledge defined by our knowledge representation may affect how the agent experiences the environment. Code is avaialbe at https://github.com/H-IAAC/cst-episode-idea

World models represent a paradigm shift in generative AI, pursuing predictive understanding and controllable simulation of environments in a structured and generalizable way. We present World Machine, a generative world-modeling architecture for time series. It is a transformer-based architecture with latent states that enables adaptation to different amounts of observed data and contexts. This shows an improvement over traditional transformers, which have a computational and memory cost that scales quadratically with the context. Experiments on a proposed synthetic dataset, Toy1D, validate the approach's feasibility, demonstrate capabilities not found in conventional transformers, and highlight the contributions of each component of the training protocol.

The ability to communicate with robots using natural language is a significant step forward in human-robot interaction. However, accurately translating verbal commands into physical actions is promising, but still presents challenges. Current approaches require large datasets to train the models and are limited to robots with a maximum of 6 degrees of freedom. To address these issues, we propose a framework called InstructRobot that maps natural language instructions into robot motion without requiring the construction of large datasets or prior knowledge of the robot's kinematics model. InstructRobot employs a reinforcement learning algorithm that enables joint learning of language representations and inverse kinematics model, simplifying the entire learning process. The proposed framework is validated using a complex robot with 26 revolute joints in object manipulation tasks, demonstrating its robustness and adaptability in realistic environments. The framework can be applied to any task or domain where datasets are scarce and difficult to create, making it an intuitive and accessible solution to the challenges of training robots using linguistic communication. Open source code for the InstructRobot framework and experiments can be accessed at https://github.com/icleveston/InstructRobot.

Unmanned Aerial Vehicles (UAVs) are increasingly deployed in complex, dynamic environments, demanding advanced control systems for safe and efficient autonomous navigation. This work explores a deep reinforcement learning approach using the Soft Actor-Critic algorithm for model-free, mapless UAV navigation. Leveraging visual inputs compressed via an autoencoder alongside embedded sensor data, we construct multimodal state representations that enable robust low-level flight control. The proposed framework is evaluated across progressively challenging scenarios—from obstacle-free settings to environments with fixed and dynamic obstacles—demonstrating effective policy generalization and stable flight behavior. Experimental results indicate that SAC enables resilient UAV navigation through continuous policy improvement, even under unstructured and previously unseen conditions.

In the emerging field of robotics, the initial development of robots handling basic tasks represented a revolutionary breakthrough. With ongoing advancements in science and technology, we anticipate the integration of artificial entities into diverse environments, performing a range of tasks autonomously. This necessitates the cultivation of intelligence and adaptability in these agents. To achieve this, agents must make informed decisions, considering environmental consequences. Motivation becomes pivotal, guiding decision-making and behavior toward fulfilling the agent's needs, drawing inspiration from human adeptness driven by motivation. This research endeavors to adapt two distinct motivation theories from human literature for the learning process of artificial agents. Employing a cognitive architecture, our investigation involves the learning process of a simulated humanoid robot, incorporating two distinct motivation systems: (i) behaviors driven by enduring needs and (ii) behaviors arising from impulses linked to temporary needs. The primary aim is to enhance comprehension of behaviors under diverse motivational frameworks within artificial cognitive agents and give insights into the direction of autonomous robots.

A Mostra Nacional de Robótica (MNR) é uma das mostras científicas de caráter nacional apoiadas pelo governo federal no Brasil. Sua temática é centrada na automação e na robótica, representantes inatos das novas tecnologias, elementos com forte capacidade para estimular o imaginário das pessoas e elementos fortemente integrados com processos modernos de ensino-aprendizagem, dialogando diretamente com diversas disciplinas dos ciclos fundamental, médio e técnico. O formato adotado pela MNR busca reunir elementos de divulgação científica típicos das atividades museológica, literária e de exposição, aliada ao uso da multimídia como estratégia de aproximação dos jovens, em sintonia com as demandas da pós-modernidade. O presente artigo apresenta um retrospecto da experiência de pouco mais de uma década de organização da MNR (2011-2022), discute os principais números e resultados alcançados, e faz uma análise da eficácia da ação na divulgação científica e como política pública, e aponta caminhos para o crescimento desta atividade no Brasil.

This paper introduces the Episode Tracker Module, an encoding mechanism that tracks sensory information through space and time, building up high-level semantic representations called episodes. This module is aimed to extend the Cognitive Systems Toolkit (CST) as a reusable framework for building different cognitive models for episode detection. We created two instances of the episode tracker with two different mechanisms for identifying property categories (geographical regions). Each mechanism correctly induced a different episode detection dynamic. Overall, the Episode Tracker architecture provides a robust and flexible framework for episode detection.

Integrating robots into our daily lives, once a distant dream, is gradually becoming a reality, surpassing our initial expectations. Today, we aspire for these robots to not only perform rudimentary tasks but to emulate human behavior, and in some aspects, even exceed it. The realm of research dedicated to achieving human-like competencies in robots has given rise to the fields of Developmental and Cognitive Robotics. These domains find their foundation in cognitive architectures and insights from human development. Despite the substantial progress in these fields, a conspicuous gap exists in the literature related to the evaluation of cognitive architectures and the advanced capabilities exhibited by robots. Recognizing this void, we aim at establishing a bridge between the insights gleaned from human developmental theories and the potential applications in robotics. Central to our investigation is the notion that learning follows a cumulative trajectory of escalating complexity. Consequently, our focus centers on the early stages of human development, particularly within the realm of children aged 0 to 2 years. Drawing inspiration from Piaget’s constructivist theory aligned with empirical studies in the Developmental Robotics domain, we unveil a framework that facilitates the classification of these studies. In light of this, we curate a series of progressive experiments, mirroring the motor and cognitive growth exhibited by children from birth to two years of age, to be conducted with robots. We also described a methodology for designing these experiments considering the robotics aspects.

Humans have needs motivating their behavior according to intensity and context. However, we also create preferences associated with each action's perceived pleasure, which is susceptible to changes over time. This makes decision-making more complex, requiring learning to balance needs and preferences according to the context. To understand how this process works and enable the development of robots with a motivational-based learning model, we computationally model a motivation theory proposed by Hull. In this model, the agent (an abstraction of a mobile robot) is motivated to keep itself in a state of homeostasis. We introduced hedonic dimensions to explore the impact of preferences on decision-making and employed reinforcement learning to train our motivated-based agents. In our experiments, we deploy three agents with distinct energy decay rates, simulating different metabolic rates, within two diverse environments. We investigate the influence of these conditions on their strategies, movement patterns, and overall behavior. The findings reveal that agents excel at learning more effective strategies when the environment allows for choices that align with their metabolic requirements. Furthermore, we observe that incorporating pleasure as a component of the motivational mechanism affects behavior learning, particularly for agents with regular metabolisms depending on the environment. Our study also unveils that, when confronted with survival challenges, agents prioritize immediate needs over pleasure and equilibrium. These insights shed light on how robotic agents can adapt and make informed decisions in demanding scenarios, demonstrating the intricate interplay between motivation, pleasure, and environmental context in autonomous systems.

Recent advancements in AI and deep learning have created a growing demand for artificial agents capable of performing tasks within increasingly complex environments. To address the challenges associated with continuous learning constraints and knowledge capacity in this context, cognitive architectures inspired by human cognition have gained significance. This study contributes to existing research by introducing a cognitive-attentional system employing a constructive neural network-based learning approach for continuous acquisition of procedural knowledge. We replace an incremental tabular Reinforcement Learning algorithm with a constructive neural network deep reinforcement learning mechanism for continuous sensorimotor knowledge acquisition, thereby enhancing the overall learning capacity. The primary emphasis of this modification centers on optimizing memory utilization and reducing training time. Our study presents a learning strategy that amalgamates deep reinforcement learning with procedural learning, mirroring the incremental learning process observed in human sensorimotor development. This approach is embedded within the CONAIM cognitive-attentional architecture, leveraging the cognitive tools of CST. The proposed learning mechanism allows the model to dynamically create and modify elements in its procedural memory, facilitating the reuse of previously acquired functions and procedures. Additionally, it equips the model with the capability to combine learned elements to effectively adapt to complex scenarios. A constructive neural network was employed, initiating with an initial hidden layer comprising one neuron. However, it possesses the capacity to adapt its internal architecture in response to its performance in procedural and sensorimotor learning tasks, inserting new hidden layers or neurons. Experimentation conducted through simulations involving a humanoid robot demonstrates the successful resolution of tasks that were previously unsolved through incremental knowledge acquisition. Throughout the training phase, the constructive agent achieved a minimum of 40% greater rewards and executed 8% more actions when compared to other agents. In the subsequent testing phase, the constructive agent exhibited a 15% increase in the number of actions performed in contrast to its counterparts.

A presente tese objetivou investigar a psicogênese das condutas cognitivas na atividade de programação realizada por 21 crianças de 4-12 anos de idade envolvidas em resolver problemas para fazer movimentar um robô concreto sem a utilização do software de programação na tela de um computador. Para avaliar o estádio do desenvolvimento intelectual, bem como o desenvolvimento das estruturas lógicas elementares de classificação e de seriação, as noções iniciais de objeto, causalidade e espaço, foram realizadas nove provas cognitivas selecionadas diretamente das obras de Piaget. Para a investigação dos processos cognitivos envolvidos na construção do algoritmo foram elaborados e aplicados doze problemas para serem resolvidos pelos sujeitos com o uso da ferramenta Kids First Coding & Robotics produzida pela empresa Thames e Kosmos. O procedimento de resolução dos experimentos foi gravado em vídeo e analisado com o suporte do software MAXQDA®. A análise, com base na Epistemologia Genética de Jean Piaget e na Microgênese Cognitiva de Bärbel Inhelder, possibilitou definir níveis de construção do algoritmo de programação e as capacidades cognitivas envolvidas na atividade de programação relacionadas com o estádio do desenvolvimento infantil. Estes resultados apontam contribuições da macrogênese e da microgênese para refletir acerca do desenvolvimento cognitivo infantil utilizando ferramentas tecnológicas de programação, além de indicar contribuições para o ensino de programação com crianças.

Unmanned Aerial Vehicles (UAVs) have gained significant attention in various domains due to their versatility and potential applications. Effective control of UAVs is crucial for achieving desired flight behaviors and optimizing their performance. This paper presents a comprehensive exploration of learning-based approaches for controlling UAVs with fixed-rotors and tiltrotors, specifically focusing on the Proximal Policy Optimization (PPO) and Twin-Delayed Deep Deterministic Policy Gradient (TD3) algorithms. The study aims to compare and evaluate the efficacy of these two state-of-the-art reinforcement learning algorithms in controlling UAVs with varying designs and control complexities. By utilizing PPO and TD3, we address the challenges associated with maneuvering UAVs in dynamic environments and achieving precise control under different flight conditions. We conducted extensive simulations to assess the performance of PPO and TD3 algorithms in diverse UAV scenarios, considering multiple design configurations and control requirements. The evaluation criteria encompassed stability, robustness, trajectory tracking accuracy, and control efficiency. Results demonstrate the suitability and effectiveness of both PPO and TD3 in controlling UAVs.

Treinar robôs para aprender políticas complexas tem se mostrado um desafio monumental. Nesse contexto, o Aprendizado por Imitação (IL) tem como foco extrair políticas de referência de um especialista e transferi-las para robôs com a máxima fidelidade possível, geralmente através do Aprendizado por Reforço Profundo (DRL). Este trabalho apresenta um novo processo de imitação para robôs bípedes, composto por três fases distintas: i) extração de poses de especialistas humanos a partir de vídeos; ii) geração de trajetórias de referência de movimento para o robô; e iii) treinamento do robô utilizando DRL para adaptar os movimentos considerando a anatomia e dinâmica específicas do robô. Nos experimentos conduzidos em um ambiente simulado, um robô humanoide foi capaz de chutar uma bola a uma distância de 1 metro, utilizando como referência vídeos de movimentos similares realizados por humanos e extraídos do YouTube.

The ability to automatically learn movements and behaviors of increasing complexity is a long-term goal in autonomous systems. Indeed, this is a very complex problem that involves understanding how knowledge is acquired and reused by humans as well as proposing mechanisms that allow artificial agents to reuse previous knowledge. Inspired by Jean Piaget's theory's first three sensorimotor substages, this work presents a cognitive agent based on CONAIM (Conscious Attention-Based Integrated Model) that can learn procedures incrementally. Throughout the paper, we show the cognitive functions required in each substage and how adding new functions helps address tasks previously unsolved by the agent. Experiments were conducted with a humanoid robot in a simulated environment modeled with the Cognitive Systems Toolkit (CST) performing an object tracking task. The system is modeled using a single procedural learning mechanism based on Reinforcement Learning. The increasing agent's cognitive complexity is managed by adding new terms to the reward function for each learning phase. Results show that this approach is capable of solving complex tasks incrementally.

Humans have needs motivating their behavior according to intensity and context. However, we also create preferences associated with each action's perceived pleasure, which is susceptible to changes over time. This makes decision-making more complex, requiring learning to balance needs and preferences according to the context. To understand how this process works and enable the development of robots with a motivational-based learning model, we computationally model a motivation theory proposed by Hull. In this model, the agent (an abstraction of a mobile robot) is motivated to keep itself in a state of homeostasis. We added hedonic dimensions to see how preferences affect decision-making, and we employed reinforcement learning to train our motivated-based agents. We run three agents with energy decay rates representing different metabolisms in two different environments to see the impact on their strategy, movement, and behavior. The results show that the agent learned better strategies in the environment that enables choices more adequate according to its metabolism. The use of pleasure in the motivational mechanism significantly impacted behavior learning, mainly for slow metabolism agents. When survival is at risk, the agent ignores pleasure and equilibrium, hinting at how to behave in harsh scenarios.

Controlar os movimentos de robôs projetados para atuar com ferramentas e em ambientes feitos para humanos tem se mostrado uma tarefa desafiadora. Essa dificuldade tem estimulado a investigação de novos métodos de controle que independam do modelo dinâmico dos robôs. A robô humanoide Marta, projetada e construída pelo grupo de trabalho, apresenta características construtivas diferenciadas: pés pequenos, pontas dos pés articuladas e junta esférica na cintura. Tendo em vista a arquitetura deste robô, o presente trabalho apresenta um controlador livre de modelo para a marcha de robôs humanoides composto por dois blocos desacoplados: i) um controlador clássico para a articulação esférica da cintura que foca na estabilidade do robô e ii) um controlador de trajetória para marcha baseada na Série Truncada de Fourier com parâmetros sintonizados por algoritmo genético. Os resultados mostram que a arquitetura mecânica e a estratégia de controle desacoplado têm juntas potencial para conferir robustez adicional ao caminhar de robôs bípedes em relação às técnicas convencionais.

: This article proposes CogToM-CST, an implementation of a Theory of Mind (ToM) model using the Cognitive Systems Toolkit (CST). Psychological research establishes that ToM deficits are usually associated with mind-blindness, the inability to attribute mental states to others, a typical trait of autism. This cognitive divergence prevents the proper interpretation of other individuals’ intentions and beliefs in a given scenario, typically resulting in social interaction problems. Inspired by the psychological Theory of Mind model proposed by Baron-Cohen, this paper presents a computational implementation exploring the usefulness of the common concepts in Robotics, such as Affordances, Positioning, and Intention Detection, to augment the effectiveness of the proposed architecture. We verify the results by evaluating both a canonical False-Belief task and a subset of tasks from the Facebook bAbI dataset.

A large number of robotic kits is commercially available for children education nowadays. Regarding the programming possibilities of these kits, they are usually divided into three categories: i) textual languages based kits, ii) digital blocks based kits and iii) tangible blocks based kits. In tangible blocks kits, the programming instructions are represented by physical - touchable - elements like blocks, cards or buttons. Since these are concrete objects, they are usually more playful and accessible for young children mainly in the concrete operational stage. Another relevant aspect in robotic kits is the possibility for children to follow the code execution at runtime. This process may be crucial for comprehending the programming process and for developing debugging abilities. In tangible programming based kits, this visual tracking is far more complex than in textual or digital blocks based kits since it typically requires the transmission of information between a lot of distinct physical pieces. The present work proposes and discusses the implementation of a digital electronic circuit with bluetooth communication designed for educational robotics kit and capable to provide a visual feedback. Results demonstrate that the proposed circuit is functional and feasible for embedding in robotic kits at a low cost.

Este trabalho apresenta um agente cognitivo baseado no modelo CONAIM (Conscious Attention-Based Integrated Model) que pode aprender incrementalmente procedimentos para a realização de experimentos inspirados nos três primeiros subestágios sensório-motores da Teoria de Jean Piaget. O agente foi implementado usando o Cognitive Systems Toolkit (CST) e validado com robôs móveis em ambientes simulados. Os resultados demonstram que a abordagem pode resolver experimentos complexos de rastreamento de objetos através do aprendizado incremental.

In this work we address the use of visualization tools for developing intelligent agents built with Cognitive Architectures (CAs). Unfortunately, there are just a few discussions about this issue in the CAs literature. We start by bringing a summary on how this topic is currently being addressed within the community and further introduce our efforts in building visualization tools for both debugging and aiding the understanding of the inner mechanisms of a CA. For this, we propose the MindViewer, a visualization tool for monitoring and debugging CAs constructed with CST, the Cognitive Systems Toolkit. MindViewer encompasses a variety of charts that are used to monitor the inner details of the agents’ mind. Its Web version allows inspect even complex data structures and customize their generated graphics. We exemplify their capabilities through a experimental application based on a robot simulation inspired in Iowa Gambling Task.