Micro-gesture classification depends on short, low-ampli- tude, and class-dependent cues, making global ensemble weights brittle when many frozen experts are available. We present CLOG-Fusion, a 416-parameter class-wise log-opinion fusion head for The 4th MiGA-IJCAI Challenge Track 1: Micro-gesture Classification, an official Kaggle competition. The implementation uses a SwinT-enhanced expert pool built from iMiGUE-finetuned MM-Gesture SwinT backbones plus auxiliary motion, MA-52-pretrained, and PoseC3D skeleton experts. The final YUV submission achieved a 0.74243 official-test leaderboard score and ranked second on the completed leaderboard.
Several interpretability methods for speech emotion recognition (SER) highlight salient features that drive the model’s decision, but they rarely assess whether learned representations align with human expert knowledge of how emotion is expressed in speech to ensure AI decisions match human values, safety, and expectations. To address this gap, we introduce a new framework that aligns model-inferred acoustic concepts with human expert-referenced psychoacoustic concepts. Expert concepts are formalized as temporally grounded abstractions of affective vocal expression, while model-inferred concepts are extracted from internal representations using concept relevance propagation (CRP). Experiments on SER models show that the proposed temporal alignment framework, TempA, distinguishes prediction errors, augments standard performance metrics with interpretable concept-level diagnostics, and reveals limitations in existing expert acoustic–affective abstractions. TempA offers new insight into the behavior of the SER model and supports validating its learned representation by measuring its alignment with human domain expertise.
This research expands upon our previous work on few-shot learning for multiclass classification on tabular data. Expanding on the FSL-LFMG framework, which integrates latent feature augmentation, advanced STUNT multitasking generation, and Prototypical Networks, we aim to illustrate the architecture’s flexibility in adapting to new concept tasks. Our prior research demonstrated the efficacy of FSL-LFMG in few-shot multiclass classification, outperforming traditional Tree Ensemble models and One-vs-the-rest classifiers. This research further investigates the framework’s generalization capabilities by applying it to a new concept tasks, specifically predicting customer loyalty levels based on an existing NPS segmentation model. Experimental results demonstrate substantial performance in both 1-shot and 5-shot configurations, attaining mean test accuracies of 83.95
In recent years, Deep Learning (DL) models have shown excellent results in various fields, including healthcare, but their “opaque” nature limits their trust and adoption in critical contexts. In this work, we present XAIMed2.0, an application designed to support healthcare professionals in interpreting the decisions of CNN models applied to medical image analysis. The application integrates several eXplainable AI (XAI) techniques to offer different visual explanations. The application integrates multiple eXplainable AI (XAI) techniques to generate visual explanations. Specifically, it employs several saliency map methods (Grad-CAM, Grad-CAM++, RISE, and LIME) and merges them into a cumulative visualization that highlights the regions of highest consensus across methods. To complement the visual explanations, the system uses a Large Language Model (LLM) to generate textual explanations in natural language, capable of contextualizing the results, highlighting discrepancies, and improving clinician understanding.
Pre-trained computer vision models are readily available and may easily be downloaded and applied to new problems using transfer learning. The main barrier to wider uptake of computer vision models is now the availability of annotated training data, particularly for specialist tasks for which no annotated set exists. The human process of hand-annotating images is expensive, slow, and error-prone. This paper presents methods for automatically generating annotated training data for computer vision models, including generating synthetic data using computer games engine software and bootstrapping large annotation sets from a small number of manual annotations. The role of image augmentation is also explored, and we show how to produce life-like augmentations in synthetic data and how augmentation improves automated annotation techniques. We call this process Image Engineering and present experimental results and illustrative case studies.
Money laundering is the process of disguising illicit funds (“dirty money”) as legitimate assets. As the third-largest criminal industry globally, it poses significant threats to economic stability and national security. To counter this, financial institutions implement anti-money laundering (AML) frameworks, which involve customer identity verification and risk assessment, transaction monitoring to detect suspicious activity, and reporting suspicious transactions to regulators. This article focuses on the transaction monitoring component of AML. We propose a Graph Convolutional Network (GCN) model enhanced with node2vec embeddings to classify transactions as legitimate or suspicious. Unlike prior work, our model is fine-tuned using evaluation objectives aligned with current industry practice. Furthermore, we experimentally demonstrate how the class imbalance problem, which is typical of AML data, adversely impacts classification performance. We present comprehensive experiments to demonstrate and justify how important parameters of the model were optimized and selected, enabling reproducibility and future benchmarking. We also discuss how the proposed model can rank transaction alerts by offense likelihood to expedite manual investigations. Finally, experimental results show that the proposed GCN model outperforms traditional machine learning techniques and prior works that used the same dataset.
Speech-language models map continuous acoustic streams to discrete language model computations, but the acoustic encoder, speech-token interface, and LLM do not share a common clock. This mismatch creates a temporal grounding problem: a model may produce a correct transcript, answer, summary, or citation while failing to indicate which audio time steps support each public output. We survey this problem as LLM-based time-step alignment (TSA-LLM). The survey reframes speech-text temporal alignment as an interface problem between audio time, linguistic units, and LLM steps. Using the recent literature reviewed here, we organize prior and emerging approaches into six families: monotonic ASR alignment scaffolds, forced alignment and long-form timestamping, text-aligned speech-token interfaces, LLM-based forced alignment and word timestamping, timestamp-grounded reasoning and captioning, and retrieval or contextual-biasing methods for rare entities. We then synthesize these methods into an alignment-contract view in which each grounded LLM output carries role, provenance, audio support, uncertainty, and streaming commitment status. The paper does not report a new benchmark result; instead, it provides a survey taxonomy, a design vocabulary, and an evaluation checklist for future work on trustworthy speech LLMs.
Interpreting the human visual system as a cascading sequence of an information system blocks, we, relying on the experience of developing information technologies (data encoding/decoding algorithms) and using specialized neurophysiological knowledge, have developed a model of an neuromorphic system, which in turn maximally imitates the perceptual mechanisms of the visual system. Essentially, we develop a neuromorphic system for processing data streams (registered counts) by extending Marr’s primal sketch paradigm with encoder blocks and implementing it at the algorithmic level. This article presents the results of our previous work in a more detailed and substantiated manner, as well as some new results. The results include a number of data representations internal to the neuromorphic system and algorithmic procedures designed to process them. Most of the obtained theoretical results are illustrated by the results of computer simulation.
The Traveling Salesperson Problem (TSP) is maybe one of the most important 𝒩𝒫 -complete optimization problems. Due to its many local optima, simple local searches like the (1+1) EA or the Randomized Local Search (RLS) are not suitable to solve the TSP. Our past research has shown that plugging Frequency Fitness Assignment (FFA) into such local searches can significantly improve their chance to reach the global optima. We showed that FRLS, i.e., RLS with FFA plugged-in, can find the optima of many more instances of the TSP than pure local search. In these prior works, we concluded that the reason for this good performance probably is ability of FFA to escape from local optima. However, truly exploring how FFA impacts the search is complicated. For most of the available TSP benchmark instances, the number #L of local optima, the sizes BL of their basins of attraction, as well as the overall number M of different tour lengths are unknown. We believe that these features will have a strong impact on the performance of RLS, FRLS, as well as all other heuristics for the TSP. Yet, determining them is not feasible either, because this would be even much more complicated than solving the instances to optimality. Knowing M , for example, does not just require to finding the globally optimal solution, but one solution corresponding at each and every possible tour length. On any problem of non-trivial size an structure, this is impossible. In this work, we pursue another route: We generate new small-scale instances for which the exact values of these parameters can be known. This allows us to explore the performance of both RLS and FRLS in settings where we know all problem instance parameters that could possibly affect the algorithms. We can therefore, for the first time on the TSP, clearly observe and confirm the ability of FRLS to escape from local optima and we can support the claim that M has a substantial impact on its performance with very conclusive evidence.
Track 2 of the 4th EI-MiGA-IJCAI Challenge studies multimodality-based online micro-gesture recognition from long video sequences. The official task requires systems to detect and classify spontaneous micro-gesture events that appear sparsely and in arbitrary order, and it allows skeleton, RGB or multimodal inputs. This paper presents our skeleton-only submission: a lightweight two-stage candidate refinement framework. The first stage trains a skeleton temporal detector to generate reliable base predictions from frame-level skeleton features. To compensate for missed gestures, a low-threshold candidate pool is further generated. Instead of directly merging all low-threshold candidates, we introduce a candidate-level Add-on MLP refiner that decides whether an extra candidate should be added to the final prediction set based on its confidence, duration and relationship with base predictions. On our validation split, the proposed add-on refinement improves the event-level F1 score to 0.5491. The final Kaggle test submission achieved a public leaderboard score of 0.35302. The results indicate that candidate-level refinement is an effective and low-cost strategy for improving skeleton-based online micro-gesture recognition under limited training data.
Deep learning and computer-aided detection (CAD) systems are increasingly vital for the early detection of lung cancer, where timely diagnosis can substantially improve patient survival. In this study, we present a two-phase deep learning framework for automated lung nodule segmentation using the Luna-16 dataset. In the first phase, the LungQuant algorithm was employed to segment the lung parenchyma, achieving an average Dice Similarity Coefficient (DSC) of 90
Evolution relies fundamentally upon two intertwined processes: speciation, where new reproductively isolated species arise from ancestral ones, and subsequent adaptation, where these isolated species evolve increasingly complex traits. This research presents an environment for simulating both of these dynamics using a basic genetic algorithm and simple simulated agents. First, we show that allopatric speciation is achievable without a priori knowledge of the environment. In our simulations, agents utilize only secondary, fitness-neutral characteristics to guide reproductive choices, creating reproductively isolated species without direct fitness comparison. This method led to a more complete separation of species than previously achieved. In addition to simulating allopatric speciation, we show that a simple genetic algorithm is capable of evolving complex features despite detrimental intermediate mutations. Our simulations demonstrate that the evolution of complex traits is feasible, though highly sensitive to the number of detrimental mutations required before such traits confer a fitness advantage. We then explore the effects of a combined simulation, allowing both speciation and complex trait evolution to take place, resulting in natural selection and competition between species without explicit fitness information.
Landscape analysis and automated algorithm selection and tuning are the subject of intense ongoing research in multi-objective optimization. Landscape analysis captures the characteristics of a problem’s landscape through numerical features. This provides machine learning models with valuable information to make appropriate decision. In addition to landscape analysis, it is also important to quantify algorithms’ component characteristics, independently of the characteristics of the problem being solved, in order to analyze and understand their relationship to performance. One of such key components is crossover. Crossover has been the focus of numerous studies. Its effectiveness depends on the operator’s properties to mix information, the specific characteristics of the problem, and the diversity of the population, influenced by the dynamics of the algorithm. This study focuses on binary representations and introduces a method to examine the relationship between crossover features and the performance of a multi-objective evolutionary algorithm on problem subclasses with random and neighbor patterns of variable interactions. The aim is to identify the crossover features relevant to performance in each problem subclass through regression models.
In this work, we propose an innovative anomaly detection platform, called METAL that integrates Multimodal-feature Extraction (ME) and Transformer-based AutoEncoder (TA) with a Location (L) classifier. Our contributions to this work are six-fold: (1) develop a signal averaging method to remove unrelated noise from the raw sensor data to improve the quality of signals; (2) extract the meaningful signal properties from three vibration signal directions using ME methods, fuse these properties together, and then reduce data dimensionality using PCA to generate a refined PCA feature set; (3) enhance a TA model to learn pump behavior from the extracted PCA feature set to detect anomalous behavior; (4) integrate a location classifier into our enhanced TA model in the training process to highlight distinctions between locations and provide feedback for enhancing performance within the AutoEncoder framework; (5) conduct an extensive experimental case study on the Stonecutters Island Sewage Treatment Works located in Hong Kong, in which we reconstruct the vibration signals of the pumps and set a threshold to detect anomalies based on reconstruction error. Having labeled data on the dates when the pumps were operating abnormally, we are able to obtain the accuracy metrics on the anomaly detection performance of the METAL pipeline. METAL achieves MCC/F1 scores of 0.961/0.972 on 1 location, 0.925/0.936 on 2 locations, and 0.768/0.821 on 6 locations that outperform the existing approaches; and (6) design and implement a web-based prototype for a Sewage Pump Monitoring System hosting the METAL pipeline, providing an interactive interface for future use.
Multimodal emotional dialogue generation (EDG) has benefited from increasingly powerful fusion architectures, yet many systems still adopt a text-centric and statically fused multimodal pipeline and fail to fully exploit audio–visual affective cues whose salience varies across dialogue contexts. We present RaMRA, a richness-aware modality-role assignment framework that estimates the semantic–sentiment informativeness of text, audio, and vision and dynamically assigns instance-level dominant/auxiliary roles. The dominant stream serves as an interaction hub for role-conditioned cross-modal attention, while response generation remains anchored in the textual context. Experiments on IEMOCAP and MELD demonstrate that RaMRA consistently improves fluency, semantic alignment, and emotion consistency over strong baselines, underscoring the importance of adaptive multimodal affect perception for building emotionally intelligent dialogue agents.
Micro-gesture classification is challenging due to subtle motion patterns, short durations, and large intra-class variations. Although RGB and skeleton modalities are commonly used, discriminative skeleton representations remain insufficiently explored. Specifically, skeleton features may contain task-irrelevant variations caused by performer-specific styles, keypoint noise, and indirect visual disturbances. Meanwhile, joint and bone streams are often fused without explicit semantic coordination, limiting their complementary potential.To address these issues, we propose a skeleton representation learning framework for micro-gesture recognition. Specifically, IB-Memory suppresses task-irrelevant variations and distills category-shared predictive semantics. In addition, JBMA aligns the prediction distributions of joint and bone streams via bi-directional KL divergence, enhancing their semantic complementarity. Experiments on the MiGA 2026 Challenge show that our method achieves 3rd place, validating the effectiveness of skeleton feature purification and joint-bone mutual alignment.
Emotional intelligence, the capacity of machines to perceive, interpret, and respond to human affective states, demands a holistic understanding of behavior that spans subtle body movements, spoken language, and social interaction. This paper presents the proceedings and position of The First Joint Workshop on Human Behavior Analysis and Interaction for Emotional Intelligence at IJCAI 2026, bringing together with the 4th MiGA Challenge. The competitive MiGA challenge attracted 66 registered participants from worldwide institutions across three tracks. Beyond the competition session, the contributed workshop papers in the other two sessions further highlight recent advances in emotional intelligence. The Explainability in Emotional Intelligence session features papers on interpretability-driven speech emotion recognition and hate speech detection, as well as trustworthiness in speech language models. The Affective Interaction and Social Computing session focuses on socially grounded and safety-critical topics, including richness-aware emotional dialogue generation and clinical safety auditing of implicit sycophancy in mental health dialogue systems. Together, these contributions position the field toward trustworthy and socially aware emotional intelligence, while exposing persistent open challenges in evaluation rigor, domain adaptation, and clinical deployment.
The adverse effects of hate speech, defined as derogatory expressions that incite hatred based on immutable characteristics, have led to increasing efforts to regulate its proliferation on social media platforms. Beyond detecting hate speech, it is important to improve explainability of models to help understand the underlying mechanisms of hate speech. In this work, we present HateScan, a multimodal web application for detecting hate speech in spoken (hatespeech) and written (hatetext) content. Hand-crafted acoustic and linguistic features are used for classification, visualization, and analysis of class-wise differences. Furthermore, we propose sentence-level context modeling for hatetext detection to improve classification performance and provide deeper insights by identifying hateful segments in textual content. The application also supports sentence-level linguistic feature analysis to help examine feature variations across individual sentences. Overall, HateScan supports the understanding of individual model decisions and promotes more transparent hate speech classification.
