Tokenization inherently affects the way information is processed within Large Language Models (LLMs), with dramatic implications for performance. Common strategies such as Byte Pair Encoding (BPE) used in the likes of GPT-2 tend to output linguistically inconsequential segments simply because they operate statistically, posing a risk of impeding predictability of models (quantified by perplexity). This work explores tokenization techniques to get lower perplexity. We present AMEST, a new hybrid tokenization that leverages both linguistic insight (data-driven morphological segmentation) and statistical soundness (probabilistic sampling of Unigram), with primary emphasis on favouring morphemes for familiar terms and sampling to handle unknown ones. We measured by training models of GPT-2 Small from scratch across eight varied sets of data to demonstrate that AMEST reliably registers significant perplexity decreases, improvements in convergence and competitive throughput vs. GPT-2 BPE and Baseline Unigram tokenizer. Our analysis, including ablation studies and cross-domain evaluations, confirms that the synergy between morphological awareness and probabilistic fallback is key to these improvements.
Thanks to recent advances in deep learning based algorithms, humanoid social robots are increasingly exhibiting human-like behaviors. In this context, the analysis of soft biometrics, particularly emotion recognition, is crucial for enhancing communication between social robots and humans, facilitating emotion-aware dialogues. In light of these considerations, we propose a multimodal emotion recognition system tailored for social robotics applications. The system processes both video and audio data to classify the emotion in one among six different classes, employing 3D convolutional operations that eliminate the need for transformer-based architecture, effectively reducing the model's size and making the network able to run over low power embedded devices mounted directly on board of the robot. The proposed approach was trained on the CREMA-D dataset and demonstrates impressive performance when compared to video-only and audio-only counterparts, outperforming state-of-the-art methods both unimodal and multimodal.
Data valuation and monetization are becoming increasingly important across domains such as eXtended Reality (XR) and digital media. In the context of 3D scene reconstruction from a set of images -whether casually or professionally captured - not all inputs contribute equally to the final output. Neural Radiance Fields (NeRFs) enable photorealistic 3D reconstruction of scenes by optimizing a volumetric radiance field given a set of images. However, in-the-wild scenes often include image captures of varying quality, occlusions, and transient objects, resulting in uneven utility across inputs. In this paper we propose a method to quantify the individual contribution of each image to NeRF-based reconstructions of in-the-wild image sets. Contribution is assessed through reconstruction quality metrics based on PSNR and MSE. We validate our approach by removing low-contributing images during training and measuring the resulting impact on reconstruction fidelity.
Fungi are living organisms that inhabit the soil and can cause severe root diseases in various crops. These diseases result from the interaction between the pathogen, the host, and the biotic and abiotic components of the soil. These fungi are generally resilient and remain inactive in the absence of the host plant. However, in the presence of a vulnerable host in the rhizosphere or the absence of adequate nutrients, these resilient structures infect the plant. The fungi in the soil can spread to other plants, in some cases without inducing symptoms that may be visible, and it can survive in crop residues. Therefore, this set of characteristics of biology, ecology, and resilience in the soil results in a complex management situation in cases of root diseases caused by fungi, which mainly cause root rot. Analysis methods are usually based on manually scoring the severity of the disease. The method proposed in this work was to create a system capable of detecting the presence of fungi that cause root rot leveraging a state-of-the-art convolutional neural network model, a YOLO-based object detection framework, and to evaluate its reliability and accuracy by comparing it with other recent work on predicting the state of plant health. The proposed network was trained with 125 samples from 15 bean genotype lines. We obtained a confidence level of 85
Unsupervised object discovery, the task of identifying and localizing objects in images without human-annotated labels, remains a significant challenge and a growing focus in computer vision. In this work, we introduce a novel model, DADO (Depth-Attention self-supervised technique for Discovering unseen Objects), which combines an attention mechanism and a depth model to identify potential objects in images. To address challenges such as noisy attention maps or complex scenes with varying depth planes, DADO employs dynamic weighting to adaptively emphasize attention or depth features based on the global characteristics of each image. We evaluated DADO on standard benchmarks, where it outperforms state-of-the-art methods in object discovery accuracy and robustness without the need for fine-tuning.
In image recognition, knowledge distillation is a valuable approach to train a compact model with high accuracy by exploiting outputs of a highly accurate large model as correct labels. In knowledge distillation, studies have shown the usefulness of data with high entropy output generated by image mix data augmentation techniques. Other strategies such as curriculum learning have also been proposed to improve model generalization by the control of the difficulty of training data over the learning process. In this paper, we explore the relationship between the learning process and data characteristics, focusing on the entropy of the output distribution and learning difficulty in knowledge distillation. To validate this relationship, we propose a method to readily generate data that yields high entropy output and adjust the degree of learning difficulty by controlling bounds and a sampling range for mix ratios in mixing images between classes. In evaluation experiments using multiple datasets on several numbers of epochs, our proposed method outperformed conventional approaches in terms of accuracy.
Pedestrian Attribute Recognition (PAR) is a fundamental task in surveillance and intelligent vision systems, aiming to identify attributes such as clothing color, gender, and accessories carried from pedestrian images. In this paper, we propose a region-aware, prompt-guided, and graph-based multi-task framework developed as a solution to the PAR 2025 Contest. Our method integrates a fully fine-tuned CLIP ViT-B/32 vision-language encoder with a Graph Attention Network (GAT)-based classifier that models inter-attribute dependencies through attention-driven message passing. The system extracts visual features from both full-body and lower-body views, computes similarity scores with a curated set of 160 handcrafted textual prompts, and feeds these semantically aligned representations into a graph-based classifier. Evaluated on the private test set, our framework achieves a mean accuracy of 69.8
Activation functions are fundamental components in neural networks, enabling non-linear transformations essential for tasks like signal processing, control systems, image analysis, economics, and robotics. They play a crucial role in facilitating processes such as noise reduction, segmentation, and decision-making across various applications. Splines offer an alternative approach to traditional activation functions (e.g., ReLU or Sigmoid), providing flexibility and adaptability to enhance function approximation. In this work a specific spline, the Piecewise Linear Fractional Function (PLRF), is introduced and proposed as a re-scoring mechanism for soft Non-Maximum Suppression (NMS) in object detection pipelines. The PLRF is parametrized with up to four hyperparameters within the range (0, 1) and the paper presents two black-box optimization techniques, GridFib and HybridNM, to refine hyperparameters. Experimental results on two different datasets indicate that the PLRF achieves higher scores compared to Greedy-NMS and Soft-NMS methods. Furthermore, the number of function evaluations needed with the proposed optimization methods reduces computational evaluations needed relative to the Bayesian optimization technique commonly used in this context.
Collaborative decision-making is essential in expert-driven image classification tasks, where individual assessments may be inconsistent or limited. We propose a task- and label-independent spatiotemporal graph neural network (STGNN) framework to model realtime interactions among human participants during group classification. The architecture combines graph neural networks (GNNs) and recurrent units to capture relational and temporal dependencies across dynamic graph sequences, with an auxiliary contrastive loss encouraging alignment among agreeing participants, coherence with chosen options and separation from alternatives. Experiments on a collaborative web platform covered five expert classification tasks of varying complexity, including cyanobacteria and diatom identification, Ki67 scoring, HER2 grading and glomerulonephritis diagnosis. From 1,369 group classification instances by 34 participants, multiple STGNN configurations were tested, varying GNN architecture, feature initialization and temporal granularity. Stratified 5-fold cross-validation showed several configurations outperforming the majority voting (MV) baseline in global top1 accuracy, with the best (GIN+GRU, T = 20) achieving 0.7757 vs. 0.7633 for MV. Improvements were also observed in complex tasks such as glomerulonephritis (0.4778 vs. 0.4167), HER2 (0.6100 vs. 0.5633), and Ki67 (0.8261 vs. 0.7993), demonstrating the potential of STGNNs for enhancing collaborative image classification.
Neural Radiance Fields (NeRFs) have transformed image-based 3D reconstruction through differentiable volumetric rendering, enabling high-quality novel view synthesis. However, their implicit volumetric nature is incompatible with the polygonal meshes needed for real-time graphics and simulation applications. The proposed model defines the volume density function as the Secant Hyperbolic Function applied to a signed distance function (SDF) representation. To enable accurate surface representation, the sharpness of the density transition is modulated by a spatially-varying parameter β (x) , which is learned through a multi-layer perceptron (MLP). Experimental results on the NeRF-Synthetic and Mip-NeRF 360 datasets demonstrate improved surface reconstruction accuracy and visual quality compared to NeRF2Mesh, highlighting the effectiveness of the proposed enhancements for efficient and high-fidelity real-time scene reconstruction.
Insect monitoring is a field of growing importance as the need to evaluate the effects of various stressors on insect populations rises. Existing methods for insect monitoring are often unsuitable for continuous monitoring of larger areas. This work presents an approach that aims to separate insect trajectories from background information in dynamic vision sensor (DVS) recordings on a new dataset. The dataset consists of approximately one hour of training data and six one-minute-long test sets of varying difficulty. A method to synthetically generate foreground-background segmentation-labeled data for this task given appropriately created source recordings is presented. To demonstrate the performance of the approach, an existing method for insect tracking in DVS data and a U-Net-based method are evaluated. The U-Net method achieves a Matthews correlation coefficient (MCC) of 0.955 detecting wasps and 0.850 detecting varied insects in front of a complex natural background. The evaluation of the existing method on the new dataset shows that it is not applicable in all use cases.
Detecting obstacles on railway tracks, such as rocks, is crucial for train safety. In this paper we propose a two-shot architecture for rocks detection: semantic segmentation is used to identify track regions, and a patch extractor is employed to guide a multi-expert system combining the decisions of a convolutional neural network (CNN) classifier and of a Vision Language Model (VLM). The former offers rock detection capability learned from the domain-specific training set, while the latter, pre-trained on millions of general image-text tuples, can recognize rocks and related concepts and distinguish them from similar object categories; these features make them complementary tools that can enhance each other's performance by combining precise expertise with adaptive generalization. As the experiments confirm, the proposed approach achieves 0.897 F1-score, outperforming the CNN classifier of 5 percentage points and the VLM of 7 percentage points, demonstrating a notable reliability in rocks detection on railway tracks.
Enhancing the logistic efficiency and safety of freight transport requires fast, reliable identification of hazardous materials (hazmat). In this work, we explore how computer vision can automate the detection and reading of hazmat number plates on freight trains and trucks. We benchmark two object detection models for hazmat localization, YOLOv11x and Faster R-CNN, across a private freight train dataset and HazTruck, our newly introduced public dataset. For reading the detected plates, we evaluated three Optical Character Recognition (OCR) methods: the widely used Tesseract, EasyOCR, and the recent vision-language model Idefics2. Integrating YOLOv11x and Idefics2 into a unified pipeline achieved the state-of-the-art performance, with over 90 https://github.com/Robust-Rail .
This paper introduces orthogonal polynomials into the framework of Kolmogorov-Arnold Networks (KANs) by replacing spline parameterized edges with two variants: (1) discrete Shmaliy polynomials and (2) the classical Legendre polynomials, while keeping nodes as input summation points. We evaluate these polynomial-based networks against the original spline-based KAN across three benchmarks. On Fashion MNIST, Shmaliy reaches 87.6
Deep vision models are now mature enough to be integrated in industrial and possibly critical applications such as autonomous navigation. Yet, data collection and labeling to train such models requires too much efforts and costs for a single company or product. This drawback is more significant in critical applications, where training data must include all possible conditions including rare scenarios. In this perspective, generating synthetic images is an appealing solution, since it allows a cheap yet reliable covering of all the conditions and environments, if the impact of the synthetic-to-real distribution shift is mitigated. In this article, we consider the case of runway detection that is a critical part in autonomous landing systems developed by aircraft manufacturers. We propose an image generation approach based on a commercial flight simulator that complements a few annotated real images. By controlling the image generation and the integration of real and synthetic data, we show that standard object detection models can achieve accurate prediction. We also evaluate their robustness with respect to adverse conditions, in our case nighttime images, that were not represented in the real data, and show the interest of using a customized domain adaptation strategy.
This work presents a method for analyzing coastal areas to extract regions of interest and identify significant events near the shore, using semantic segmentation adapted to these environments. The segmentation approach is applied to label all pixels in an image according to a predefined set of classes. Two additional classes-namely, foam and wet sand-are introduced to the typical categories used in coastal dynamics, allowing for more detailed differentiation of areas that are important for specific purposes. The resulting classifications are then analyzed, either individually or as a sequence of frames in a video, to detect the occurrence of relevant events, such as waves overtopping dikes and reaching pedestrian or vehicle areas, or to extract regions of interest, such as the intertidal zone. In particular, detecting overtopping involves selecting a critical region and monitoring when it is reached by the sea. On the other hand, extracting the intertidal zone implies processing sequences spanning several hours to track the sea's temporal changes. With this approach and the additional classes, the proposed method enables more robust detection of overtopping events and more accurate delineation of the region between high and low tides.
Recent works and challenges on pedestrian attribute recognition demonstrated the necessity to collect extensive and representative datasets and to propose effective and efficient methods based on advanced neural networks. Following on the success of the first edition, the Pedestrian Attribute Recognition (PAR) 2025 Contest, organized within CAIP 2025, is an international competition designed to evaluate advanced neural networks for recognizing pedestrian attributes. Participants are provided with the new Mivia PAR KD Dataset, which includes 106,743 newly annotated images featuring a combination of labels and pseudo-labels obtained through a knowledge distillation method for attributes such as clothing color gender, and the presence or absence of accessories like bags and hats. Competing approaches have been assessed based on the mean accuracy metric, using a separate private test set comprising over 20,000 images, distinct from the training data and not provided to any participants, in order to ensure fairness in the evaluation of results. The goal was to push the participants to effectively leverage the latest advancements in neural network technologies and enhance the scalability and real-world applicability of PAR solutions. The contest teams advanced the state of the art by proposing approaches improving computational efficiency, reducing training time, better addressing class imbalance, and incorporating effective learning procedures and data augmentation strategies. The impressive 95.4
Object detection models typically rely on a predefined set of categories, limiting their applicability in real-world scenarios where object classes may be unknown. In this paper, we propose a novel, training-free framework that enables off-the-shelf open-vocabulary object detectors (OvOD) to perform category-free detection-localizing and classifying objects without any prior category knowledge. Our approach leverages image captioning to dynamically generate descriptive terms directly from the image content, followed by a WordNet-based filtering process to extract semantically meaningful category names. These discovered categories are then embedded and matched with visual region features using a frozen OvOD model to perform detection. We evaluate our method on the COCO dataset in a fully zero-shot setting and demonstrate that it significantly outperforms strong multimodal large language model baselines, achieving an improvement of over 30 AP points. This highlights our method as a promising direction for more adaptive solutions to real-world detection challenges.
The integration of advanced artificial intelligence technologies in production lines enables an effective and safer collaboration between human workers and robotic platforms in Industry 5.0, where the focus is the reinforcement of the workers capability, improving their skills when cooperating with the robots. This novel collaboration paradigm requires a natural and efficient human-robot communication way in which speech command and gesture recognition emerge as fundamental components for enhancing collaboration and fostering adaptability in industrial environments. In this paper, we present an innovative multi-modal human-robot collaboration framework, based on speech command and gesture recognition, designed to meet the requirements of accuracy and real-time processing of an existing production line, used as test environment for the European FELICE project. The speech command recognition system performs voice activity detection and is able to reliably distinguish among a set of commands in a noisy industrial environments, by combining a Mel-spectrogram based representation for the voice with a speech recognition neural network based on a Conformer. As for the recognition of gestures, the task is performed using a one-stage detector based on MobileNetV3 SSD. The experiments, performed on datasets encompassing real, synthetic, and negative samples for speech commands as well as images acquired in a realistic use case for gesture recognition, have established the suitability of the proposed solution in challenging industrial settings.
