Reinforcement Learning (RL) provides a powerful framework for decision-making in complex environments. However, implementing RL in hardware-efficient and bio-inspired ways remains a challenge. This paper presents a novel Spiking Neural Network (SNN) architecture for solving RL problems with real-valued observations. The proposed model incorporates multi-layered event-based clustering, with the addition of Temporal Difference (TD)-error modulation and eligibility traces, building upon prior work. An ablation study confirms the significant impact of these components on the proposed model's performance. A tabular actor-critic algorithm with eligibility traces and a state-of-the-art Proximal Policy Optimization (PPO) algorithm are used as benchmarks. Our network consistently outperforms the tabular approach and successfully discovers stable control policies on classic RL environments: mountain car, cart-pole, and acrobot. The proposed model offers an appealing trade-off in terms of computational and hardware implementation requirements. The model does not require an external memory buffer nor a global error gradient computation, and synaptic updates occur online, driven by local learning rules and a broadcasted TD-error signal. Thus, this work contributes to the development of more hardware-efficient RL solutions.

This study introduces a novel approach to Handwritten Mathematical Expression Recognition (HMER), focusing on elementary school mathematical expressions. Recognizing the challenges posed by limited training data and the unique characteristics of elementary students' handwriting, we present a multiobjective optimization method tailored for small training datasets. We employ state-of-the-art HMER methods, including transformer-based and attention mechanism models, and optimize them using a custom dataset comprised of elementary school arithmetic equations. This dataset contains 1237 images and includes both horizontal and vertical equations and isolated numbers, featuring common errors in children's handwriting. Additional similar datasets are also leveraged for training augmentation. Our experimental results demonstrate the efficacy of the optimization approach, significantly improving the performance of the evaluated models in terms of expression recognition rate and inference speed. This study contributes to the field of HMER by providing an effective optimization approach for SOTA models and by introducing a specialized dataset for elementary school mathematics. The dataset is available upon request.

Artificial Intelligence in Education (AIED) implementation in underserved regions faces challenges due to limited digital infrastructure, such as restricted device and internet access. A solution to these challenges lies in AIED Unplugged, a framework designed to address these challenges by tailoring AI solutions to the specific issues prevalent in such regions. AIED Unplugged incorporates principles like Conformity, Disconnect, Proxy, Multi-User, and Unskillfulness, ensuring accessibility by aligning with existing infrastructure, operating offline, simplifying interfaces, and accommodating users' digital skills. Particularly, the framework leverages computer vision to digitalize students' activities and enable AIED-based learning on disconnected, low-cost devices, wherein object detection is crucial to identify which solution areas to digitalize. However, prior research has not assessed the technical feasibility of such applications in the context of AIED unplugged for math education. Therefore, this paper addresses the intersection of "conformity" and "disconnected" principles with an empirical analysis of handwritten equation detection on disconnected, low-cost mobile devices. By optimizing state-of-the-art algorithms for offline inference and considering device constraints, we utilize a dataset of student equations, explore YOLOv8 models, and evaluate its predictive performance. The trained model is converted to Tensorflow Lite for mobile deployment, and a testbed application assesses inference times on diverse low-cost devices, contributing valuable empirical insights to the intersection of AIED Unplugged, Computer Vision, and Education in underserved regions.

Energy storage solutions play an increasingly important role in modern infrastructure and lead-acid batteries are among the most commonly used in the rechargeable category. Due to normal degradation over time, correctly determining the battery's State of Health (SoH) and Remaining Useful Life (RUL) contributes to enhancing predictive maintenance, reliability, and longevity of battery systems. Besides improving the cost savings, correct estimation of the SoH can lead to reduced pollution though reuse of retired batteries. This paper presents a mapping study of the state-of-the-art in machine learning methods for estimating the SoH and RUL of lead-acid batteries. These two indicators are critical in the battery management systems of electric vehicles, renewable energy systems, and other applications that rely heavily on this battery technology. In this study, we analyzed the types of machine learning algorithms employed for estimating SoH and RUL, and evaluated their performance in terms of accuracy and inference time. Additionally, this mapping identifies and analyzes the most commonly used combinations of sensors in specific applications, such as vehicular batteries. The mapping concludes by highlighting potential gaps and opportunities for future research, which lays the foundation for further advancements in the field.

Sistemas Tutores Inteligente (STIs) têm sido amplamente utilizados para auxiliar no aprendizado de matemática. No entanto, a diferença na forma de inserção de soluções nos STIs, que requer o uso de um teclado, em comparação com a prática padrão de escrever à mão, pode levar a problemas de usabilidade e prejudicar a aprendizagem. Para superar essa limitação, pesquisas recentes têm explorado o reconhecimento de caracteres escritos à mão em papel como entrada para os STIs. Porém, existe uma lacuna de conhecimento em relação ao desempenho dos algoritmos de reconhecimento de dígitos avançados no contexto de operações matemáticas básicas. Este artigo compara quatro algoritmos de última geração para o reconhecimento de dígitos em problemas matemáticos de adição e subtração. Os resultados revelam que o algoritmo BTTR obteve o melhor desempenho em termos de acurácia, enquanto o algoritmo SAN apresentou um bom equilíbrio entre acurácia e velocidade de reconhecimento. Essas descobertas são relevantes para pesquisadores e desenvolvedores ao selecionar os algoritmos mais adequados para o desenvolvimento de STIs baseados em entrada escrita à mão.

With the support of Internet of Things (IoT) devices, it is possible to acquire data from degradation phenomena and design data-driven models to perform anomaly detection in industrial equipment.This approach not only identifies potential anomalies but can also serve as a first step toward building predictive maintenance policies.In this work, we demonstrate a novel anomaly detection system on induction motors used in pumps, compressors, fans, and other industrial machines.This work evaluates a combination of pre-processing techniques and machine learning (ML) models with a low computational cost.We use a combination of pre-processing techniques such as Fast Fourier Transform (FFT), Wavelet Transform (WT), and binning, which are well-known approaches for extracting features from raw data.We also aim to guarantee an optimal balance between multiple conflicting parameters, such as anomaly detection rate, false positive rate, and inference speed of the solution.To this end, multiobjective optimization and analysis are performed on the evaluated models.Pareto-optimal solutions are presented to select which models have the best results regarding classification metrics and computational effort.Differently from most works in this field that use publicly available datasets to validate their models, we propose an end-to-end solution combining low-cost and readily available IoT sensors.The approach is validated by acquiring a custom dataset from induction motors.Also, we fuse vibration, temperature, and noise data from these sensors as the input to the proposed ML model.Therefore, we aim to propose a methodology general enough to be applied in different industrial contexts in the future.

Anomaly detection is critical in the smart industry for preventing equipment failure, reducing downtime, and improving safety. Internet of Things (IoT) has enabled the collection of large volumes of data from industrial machinery, providing a rich source of information for Anomaly Detection (AD). However, the volume and complexity of data generated by the Internet of Things ecosystems make it difficult for humans to detect anomalies manually. Machine learning (ML) algorithms can automate anomaly detection in industrial machinery by analyzing generated data. Besides, each technique has specific strengths and weaknesses based on the data nature and its corresponding systems. However, a large portion of the existing systematic mapping studies on AD primarily focus on addressing network and cybersecurity-related problems, with limited attention given to the industrial sector. Additionally, the related literature do not cover the challenges involved in using ML for AD in industrial machinery within the context of the IoT ecosystems. Therefore, this paper presents a systematic mapping study on AD for industrial machinery using IoT devices and ML algorithms to address this gap. Our primary objective is to investigate the use of ML models for anomaly detection within an industrial setting, particularly within IoT ecosystems. The study comprehensively evaluates 84 relevant studies spanning from 2016 to 2023, providing an extensive review of AD research. Our findings identify the most commonly used algorithms, preprocessing techniques, and sensor types. Additionally, this review identifies application areas and points to future challenges and research opportunities.

This paper presents an optimization and multiobjective evaluation of deep learning (DL) models to improve pallet localization with an RGB-D camera in scenarios of forklift insertion. To this end, we experimentally evaluate three distinct DL models: Detectron2 for combined detection and segmentation, YoloV5 for detection only, and a combination of YoloV5 and UNet for detection and segmentation. Through the automatic hyperparameter optimization process on a custom dataset, 30 configurations were selected, each demonstrating a unique trade-off between precision and speed. Out of these, three Pareto optimal models were chosen for a more detailed analysis, considering inference speed and localization errors along each orthogonal axis. The results suggest that the proposed hybrid model combining YoloV5 and UNet exhibited a promising balance of speed and accuracy, making it suitable for real-time applications. Finally, the proposed model is demonstrated for a fork insertion task in a new environment.

Handwritten Mathematical Expression Recognition (HMER) is a challenging task with many educational applications. Recent methods for HMER have been developed for complex mathematical expressions in standard horizontal format. However, solutions for elementary mathematical expression, such as vertical addition and subtraction, have not been explored in the literature. This work proposes a new handwritten elementary mathematical expression dataset composed of addition and subtraction expressions in a vertical format. We also extended the MNIST dataset to generate artificial images with this structure. Furthermore, we proposed a solution for offline HMER, able to recognize vertical addition and subtraction expressions. Our analysis evaluated the object detection algorithms YOLO v7, YOLO v8, YOLO-NAS, NanoDet and FCOS for identifying the mathematical symbols. We also proposed a transcription method to map the bounding boxes from the object detection stage to a mathematical expression in the LATEX markup sequence. Results show that our approach is efficient, achieving a high expression recognition rate. The code and dataset are available at https://github.com/Danielgol/HME-VAS

As solar generation gains an increased importance in a smart grid, an efficient control of a photovoltaic (PV) array has to be considered. However, energy efficiency of a PV array greatly depends on environmental conditions, such as uneven shading, solar irradiance and temperature. Maximum power point tracking (MPPT) algorithms aim to dynamically find an optimal operation voltage in order to compensate for changes in the environment, as well as degradation of solar panels. Besides generating a maximum amount of power, an MPPT controller aims for stability in order to avoid additional losses. Reinforcement Learning (RL) is a flexible training method that can produce a controller for a complex problem without a detailed prior knowledge of the environment. In this work we propose an approach that combines an optimized neural network, trained through deep reinforcement learning (DRL), with a classical closed-loop control. Experimental results suggest that the proposed approach outperforms a recently proposed DRL network both in terms of efficiency and stability. A compact and discrete version of the proposed controller is also evaluated and shown to further increase performance. The implemented algorithms and the RL simulation environment are made available in an open-source repository.

Spiking neural networks (SNNs) aim to replicate energy efficiency, learning speed and temporal processing of biological brains. However, accuracy and learning speed of such networks is still behind reinforcement learning (RL) models based on traditional neural models. This work combines a pre-trained binary convolutional neural network with an SNN trained online through reward-modulated STDP in order to leverage advantages of both models. The spiking network is an extension of its previous version, with improvements in architecture and dynamics to address a more challenging task. We focus on extensive experimental evaluation of the proposed model with optimized state-of-the-art baselines, namely proximal policy optimization (PPO) and deep Q network (DQN). The models are compared on a grid-world environment with high dimensional observations, consisting of RGB images with up to 256 × 256 pixels. The experimental results show that the proposed architecture can be a competitive alternative to deep reinforcement learning (DRL) in the evaluated environment and provide a foundation for more complex future applications of spiking networks.

Biological brains still far exceed artificial intelligence systems, both in terms of control capabilities and power consumption. Spiking neural networks (SNNs) are a promising model, inspired by neuroscience and functionally closer to the way neurons process information. While recent advancements in neuromorphic hardware allow energy efficient synthesis of spiking networks, the training of such networks remains an open problem. In this work we focus on reinforcement learning with sparse and delayed rewards. The proposed architecture has four distinct layers and addresses the limitation of previous models in terms of scalability with input dimensions. Our SNN is evaluated on classical reinforcement learning and control tasks and is compared to two common RL algorithms: Q-learning and deep Q-network (DQN). Experiments demonstrate that the proposed network outperforms Q-learning on a task with six-dimensional observation space and compares favorably to the evaluated DQN configurations in terms of stability and memory requirements.

Convolutional architectures have in recent years become state-of-the-art for several object detection tasks. However, these detectors have not yet been evaluated for detection and monitoring of beach areas. As some of these areas need to be continually monitored for dangerous situations, such as shark attacks, an automated system would be an effective risk control measure. The most significant and specific challenges for this problem are variable scene illumination, partial occlusion and distant camera position. In this work we present a study on three recent convolutional architectures for the task of people detection in beach scenarios. Our dataset is composed of images taken in the Boa Viagem beach, in Brazil, and is used to evaluate Faster R-CNN, R-FCN and SSD in terms of quality and speed of detection. The detectors are pretrained on a dataset containing 91 classes of objects, including people with different levels of scale and occlusion. The results suggest that the Faster R-CNN meta-architecture with the Resnet 101 feature extractor generates significantly better detections in terms of F-measure, while performing at 5.6 fps on a GTX 1080 Ti GPU.

Gesture based human–computer interaction is both intuitive and versatile, with diverse applications such as in smart houses, operating theaters and vehicle infotainment systems. This paper presents a novel architecture, combining a convolutional neural network (CNN) and traditional feature extractors, capable of accurate and real-time hand posture recognition. The proposed architecture is evaluated on three distinct benchmark datasets and compared with the state-of-the art convolutional neural networks. Extensive experimentation is conducted using binary, grayscale and depth data, as well as two different validation techniques. The proposed feature fusion-based convolutional neural network (FFCNN) is shown to perform better across combinations of validation techniques and image representation. The recognition rate of FFCNN on binary images is equivalent to grayscale and depth when the aspect ratio of gestures is preserved. A real-time recognition system is presented with a demonstration video.

Hand gestures are an intuitive way for humans to interact with computers. They are becoming increasingly popular in several applications, such as smart houses, games, vehicle infotainment systems, kitchens and operating theaters. An effective human-computer interaction system should aim at both good recognition accuracy and speed. This paper proposes a new approach for static hand gesture recognition. A benchmark database with 36 gestures is used, containing variations in scale, illumination and rotation. Several common image descriptors, such as Fourier, Zernike moments, pseudo-Zernike moments, Hu moments, complex moments and Gabor features are comprehensively compared in terms of their respective accuracy and speed. Gesture recognition is undertaken by a multilayer perceptron which has a flexible structure and fast recognition. In order to achieve improved accuracy and minimize computational cost, both the feature vector and the neural network are tuned by a multi-objective evolutionary algorithm based on the Nondominated Sorting Genetic Algorithm II (NSGA-II). The proposed method is compared with state-of-the-art methods. A real-time gesture recognition system based on the proposed descriptor is constructed and evaluated. Experimental results show a good recognition rate, using a descriptor with low computational cost and reduced size. (C) 2017 Elsevier Ltd. All rights reserved.

The mechanisms of KCl-induced enhancement in identification of individual molecules of poly(ethylene glycol) using solitary alpha-hemolysin nanoscale pores are described. The interaction of single molecules with the nanopore causes changes in the ionic current flowing through the pore. We show that the on-rate constant of the process is several hundred times larger and that the off-rate is several hundred times smaller in 4 M KCl than in 1 M KCl. These shifts dramatically improve detection and make single molecule identification feasible. KCl also changes the solubility of poly(ethylene glycol) by the same order of magnitude as it changes the rate constants. In addition, the polymer-nanopore interaction is determined to be a strong non-monotonic function of voltage, indicating that the flexible, nonionic poly(ethylene glycol) acts as a charged molecule. Therefore, salting-out and Coulombic interactions are responsible for the KCl-induced enhancement. These results will advance the development of devices with sensor elements based on single nanopores.