Recent methods for affective analysis have made progress in polarity prediction. However while they work to generate accurate natural language explanations, the transparency and user trust is limited–especially in complex, multi-aspect social media scenarios. To bridge this gap, we propose a method named EMAA(Explainable Multimodal Affective Analysis), a two-stage framework that integrates explanation generation into aspect-based sentiment analysis, combining self-training with direct preference optimization. Additionally, we develop a filter model to automatically evaluate and select high-quality explanations, thereby significantly improving the reliability of training data. Experiments on benchmark datasets show that our method outperforms the baselines in both classification accuracy and explanation quality. Furthermore, we demonstrate that high-quality explanations can improve the robustness and interpretability of sentiment classifications.

Fuzzy rough feature selection (FRFS) effectively alleviates the curse of dimensionality by eliminating redundant and irrelevant features, thereby improving model generalization. However, most existing algorithms focus on minimizing classification uncertainty, even though lower uncertainty does not necessarily imply stronger class discrimination or improved classification performance. This challenges the common assumption that uncertainty alone sufficiently captures feature relevance in pattern classification tasks. To bridge this gap, we propose a margin-aware fuzzy rough feature selection (MAFRFS) framework that explicitly incorporates structural characteristics of class distributions, namely, within-class compactness and between-class separability, into the feature evaluation process. By integrating margin-based structural cues with fuzzy rough uncertainty modeling, MAFRFS effectively guides the selection toward more separable and discriminative feature subsets. Extensive experiments reported on 23 publicly available datasets demonstrate that MAFRFS is highly scalable and more effective than FRFS. Algorithms developed under MAFRFS consistently outperform some state-of-the-art feature selection algorithms.

Micro-expressions (MEs) are brief, involuntary facial expressions that reveal genuine emotions, making them critical for applications such as lie spotting and medical diagnostics. However, existing methods often treat spotting and recognition as separate tasks, resulting in complex workflows and limited research on micro-expression analysis. Moreover, the current integrated spotting-recognition networks are mostly built upon relatively simple convolutional architectures, which are insufficient to capture spatio-temporal dynamics inherent in micro-expressions. To address these limitations, we propose MiAFormer(Micro Analysis Transformer), a transformerbased unified network for fine-grained micro-expression spotting and recognition that leverages 3D scene flow as input. First, MiAFormer is the first unified transformer framework for ME analysis, specifically designed to capture ME-specific features from scene flow integrating depth information. Second, we introduce MiRA(Micro-analysis Routing Attention) which employs routing-based dynamic sparse attention to emphasize correlations among facial regions in a content-aware manner, enabling more efficient and discriminative feature extraction. Third, experiments on CAS(ME) ${ }^{\mathbf{3}}$ demonstrate that the proposed method is effective, and ablation studies confirm the effectiveness of both MiRA and 3D integration in advancing unified micro-expression analysis.

Feature selection is crucial for fuzzy decision systems (FDSs), as it identifies informative features and eliminates rule redundancy, thereby enhancing predictive performance and interpretability. Most existing methods either fail to directly align evaluation criteria with learning performance or rely solely on nondirectional Euclidean distances to capture relationships among decision classes, which limits their ability to clarify decision boundaries. However, the spatial distribution of instances has a potential impact on the clarity of such boundaries. Motivated by this, we propose spatially-aware separability-driven feature selection (S2FS), a novel framework for FDSs guided by a spatially-aware separability criterion. This criterion jointly considers within-class compactness and between-class separation by integrating scalar-distances with spatial directional information, providing a more comprehensive characterization of class structures. S2FS employs a forward greedy strategy to iteratively select the most discriminative features. Extensive experiments on 11 real-world datasets demonstrate that S2FS consistently outperforms ten state-of-the-art feature selection algorithms in both classification accuracy and clustering performance, while feature visualizations further confirm the interpretability of the selected features.

Label distribution learning (LDL) has garnered increased attention in recent studies on label ambiguity. However, collecting complete annotations for LDL tasks is often time-consuming and labor-intensive compared to traditional learning paradigms. Therefore, designing effective incomplete LDL algorithms is crucial to broaden LDL’s application scope. In this paper, we propose a novel LDL algorithm, called Incomplete Label Distribution Learning via Label Correlation Decomposition (IncomLDL-LCD), which simultaneously learns label distributions and recovers missing description degrees of labels through label correlations. Specifically, we decompose the label correlation into sparse local label correlation and low-rank global label correlation using a soft-thresholding operator and a singular value thresholding operator, respectively. The former is utilized to capture the related label subsets necessary for reconstructing each possible label, while the latter focuses on extracting the coarse-grained semantic concepts from all labels and exploring the groupings of labels. Additionally, we develop an alternating solution with the accelerated proximal gradient descent method for optimization. Extensive experiments on 16 real-world data sets with varying degrees of missing annotations validate that our algorithm effectively handles incomplete LDL tasks and outperforms state-of-the-art algorithms.

Gait Emotion Recognition (GER) is an emerging task within Human Emotion Recognition. Skeleton-based GER requires discriminative spatial and temporal features. However, current methods primarily focus on capturing spatial topology information but fail to effectively learn temporal features from long-distance frames. Moreover, these methods are mostly sensitive to the order of sampled sequences, resulting in significant accuracy drops when sequences are randomly sampled. In order to obtain a more robust and comprehensive spatial-temporal representation of gait, we introduce the Graph-Transformer architecture into GER for the first time, proposing a novel framework named GaitCycFormer. Specifically, we designed a Cycle Position Encoding (CPE) based on the gait cycle, which explicitly segments any gait sequence into more manageable periodic units, to enhance temporal feature modeling. Additionally, we incorporate a bi-level Transformer, consisting of an Intra-cycle Transformer and an Inter-cycle Transformer to capture local and global temporal information within each gait cycle and between gait cycles respectively. Experiments demonstrate that our GaitCycFormer achieves state-of-the-art performance on popular datasets, and proves to be more reliable and robust.

Label distribution learning (LDL) is an emerging learning paradigm designed to capture the relative importance of labels for each instance. Label-specific features (LSFs), constructed by LIFT, have proven effective for learning tasks with label ambiguity by leveraging clustering-based prototypes for each label to re-characterize instances. However, directly introducing LIFT into LDL tasks can be suboptimal, as the prototypes it collects primarily reflect intra-cluster relationships while neglecting cross-cluster interactions. Additionally, constructing LSFs using multi-perspective information, rather than relying solely on Euclidean distance, provides a more robust and comprehensive representation of instances, mitigating noise and bias that may arise from a single distance perspective. To address these limitations, we introduce Structural Anchor Points (SAPs) to capture inter-cluster interactions. This leads to a novel LSFs construction strategy, LIFT-SAP, which enhances LIFT by integrating both distance and directional information of each instance relative to SAPs. Furthermore, we propose a novel LDL algorithm, Label Distribution Learning via Label-specifIc FeaTure with SAPs (LDL-LIFT-SAP), which unifies multiple label description degrees predicted from different LSF spaces into a cohesive label distribution. Extensive experiments on 15 real-world datasets demonstrate the effectiveness of LIFT-SAP over LIFT, as well as the superiority of LDL-LIFT-SAP compared to seven other well-established algorithms.

Micro-expressions are brief and subtle facial movements. Unlike macro-expressions, they are difficult to control and can reflect true emotion. Therefore, they are valuable in criminal investigation, medical care and other applications. Micro-expression recognition refers to the emotion classification of micro-expression samples. Previous approaches frequently relied on image sequences as inputs and ignored the fact that micro-expressions are activated only in local areas, introducing irrelevant noise. Additionally, some methods solely employed traditional graph models without fully exploring the complex spatiotemporal relationships between different facial regions and frames. To address this issue, we propose the method with a Graph Transformer for micro-expression recognition to more effectively learn the interrelations between facial regions and frames, thereby obtaining more discriminative features. Specifically, we develop a novel Graph Transformer with a learnable adjacency matrix for spatiotemporal learning, which better learns long-range dependencies and adaptively integrates implicit information in the graph. We select appropriate facial landmarks and calculate the optical-flow-based feature to serve as input. Finally, experiments conducted on relevant datasets have demonstrated the effectiveness of our method.

Image captioning aims to generate fluent and accurate descriptions for images. To evaluate the quality of captions, various metrics have been proposed. However, current metrics only assess captions at sequence-level, which overwhelms the distinctions between each token. Thus, existing objectives tend to treat each token equally, assigning them with identical weights in loss functions. Intuitively, key words in a caption carry the primary information and contribute more than other words to sequence-level metrics. They should be distinguished and weighted more during training. In this work, we propose to explicitly measure each word and guide the model to focus more on key words in captions. Firstly, we devise token-level CIDEr (CIDEr-T) as a new metric to quantify the importance of each word, by decomposing the sequence-level CIDEr into token-level granularity. CIDEr-T maintains consistency with CIDEr and shows the distinctions between tokens. Thus, we engage CIDEr-T scores of each token as their unique weights in the raw loss functions, which can bridge the gap between training and evaluation.

Unbalanced classification is an essential machine learning task, which has attracted widespread attention from both the academic and industrial communities due mainly to its broad applications. Evolutionary computation (EC) has contributed greatly to unbalanced classification. However, to the best of our knowledge, there have not been any comprehensive investigations on the strengths and weaknesses of alternative EC methods in addressing various challenging problems in unbalanced classification. This paper reviews the literature which utilize EC techniques for unbalanced classification, with the aim of revealing the contributions of EC to unbalanced classification, providing an overview of recent advances, and identifying limitations of existing works. In addition, we present a series of real-world applications, and identify open challenges as well as possible research directions for the future.

Deep Neural Networks have shown vulnerability to sparse adversarial attack, which involves perturbing only a limited number of pixels. Identifying the coordinates requiring perturbation in sparse attacks poses a significant computational challenge. Existing solutions predominantly rely on heuristic methods or relax the ℓ 0 -norm to the ℓ 1 -norm. In this paper, we present an efficient algorithm for conducting sparse attacks. Our algorithm factorizes the perturbation at each pixel to the product of the perturbation coordinates and the perturbation magnitudes and then optimizes them alternately. We reformulate the ℓ 0 -norm as a stochastic binary optimization problem, assuming that each pixel’s perturbation status is associated with a stochastic binary variable. This stochastic binary variable follows a Bernoulli distribution, with a parameter value that ranges from 0 to 1, signifying the probability of pixel disturbance. To tackle this stochastic binary optimization challenge, we employ an unbiased gradient estimator known as Augment-Reinforce-Merge (ARM). Once the perturbed coordinates are determined, we optimize the perturbation magnitudes with gradient descent. Furthermore, we incorporate a binary search algorithm to eliminate redundant pixels to enhance sparsity. Comprehensive experiments demonstrate the superiority of our proposed method over several state-of-the-art sparse attack methods.

Black-box attacks pose a significant challenge due to the restricted access to target model information, hindering the generation of impactful adversarial samples. This paper introduces a method that combines sparse attacks and meta-learning to alleviate the issue of low success rates in black-box attacks. SAM leverages the knowledge transfer capabilities inherent in meta-learning to augment the transferability of adversarial samples. The method integrates meta-learning with gradient-based attack techniques, effectively transforming the approach into a white-box attack. By aggregating multiple sampled models, SAM enhances the stability of adversarial samples. During meta-testing, simulated black-box attacks help mitigate gradient discrepancies across diverse models, consequently enhancing transferability. To further improve sparsity and preserve transferability, SAM incorporates a projection strategy that selectively sparsifies global adversarial perturbations. Experimental evaluations conducted on two image datasets substantiate SAM’s superiority in terms of both sparsity and attack success rate. Ablation experiments confirm the effectiveness of integrating meta-learning into the proposed method. SAM extends the applicability of generated adversarial samples, advancing the domain of adversarial attacks in scenarios with limited target model information. The proposed approach exhibits promise in enhancing the success rate of attacks while preserving sparsity, contributing to the broader understanding of black-box attacks and their implications.

Facial expression feature extraction suffers from high inter-subject variations caused by identity-related personal attributes. The extracted expression features are consistently entangled with other identity-related features, which has an influence on related facial expression tasks such as recognition and editing. To achieve high-quality expression features, a Disentangled Variational Autoencoder (DisVAE) is proposed to disentangle expression and identity features. The identity features are removed from the facial features via facial image reconstruction firstly, and then the remaining features represent expression components. Extensive experiments on three public datasets have shown that the proposed DisVAE can effectively disentangle expression and identity features, and extract expression features without the interfere of identity attributes. The high-quality expression features improve the performance of facial expression recognition and can be well applied to facial expression editing.

The susceptibility of current Deep Neural Networks (DNNs) to adversarial examples has been a significant concern in deep learning methods. In particular, sparse adversarial examples represent a specific category of adversarial examples that can deceive the target model by perturbing only a few pixels in images. While existing sparse adversarial attack methods have shown achievements, the current results of sparsity and efficiency are inadequate and require significant improvements. This paper introduces an adv-triplet loss and proposes a search attack method to attack the Face Expression Recognition (FER) system with minimal pixel perturbations. Specifically, we propose an adv-triplet loss function and utilize its gradient information to generate pixels for adversarial examples. Extensive experiments conducted on the CK+ and Oulu-CASIA datasets demonstrate the superiority of our proposed method over several state-of-the-art sparse attack methods.

Unbalanced classification has attracted widespread interest because of its broad applications. However, due to mainly the uneven class distribution, constructed classifiers are usually biased toward the majority class, and thereby perform terribly on the minority class. Unfortunately, the minority class is often the class of interest in many real-world applications. High dimensionality often further degrades the classification performance, making it more complicated to address the class imbalance issue. Genetic programming (GP) has been applied to construct classifiers, which can simultaneously select good-quality features to improve the classification performance. To handle the class imbalance issue, cost-sensitive GP classifiers treat the minority class as being more important than the majority class, but this may cause an accuracy decrease in overlapping areas where the prior probabilities of the two classes are almost the same. To date, most cost-sensitive classification methods have not been specifically investigated how the impacts of overlapping areas on cost-sensitive classifiers can be avoided. In this study, we propose a new cost-sensitive GP method, where rough set theory is employed to detect overlapping areas before training cost-sensitive classifiers for classification with unbalanced high-dimensional data. The proposed method is compared with 46 popular classification methods, including 10 GP methods and 36 non-GP methods on 14 datasets that are unbalanced and high dimensional. The experimental results indicate that the proposed method performs better than the compared methods in almost all cases.

Question-controlled Text-aware Image Captioning (Qc-TextCap), is the task of generating a distinctive scene text aware caption according to several personalized questions when given an image. However, due to the diversity of visual scene, it is hard for current Optical Character Recognition (OCR) systems to extract complete scene text sentences from the images. Besides, existing works are limited in their use of question features and visual features. In this paper, we propose a Multimodal Transformer plus Scene text clustering and Cross modal attention (MTSC) to tackle the above challenges. We devise scene text clustering to group relevant scene text pieces which are detected as separate results by current OCR systems. To better utilize the information in questions and images, we design cross modal attention to enrich the features of both modalities. We extensively evaluate our model on the two Qc-TextCap datasets and superior results are achieved when comparing to state-of-the-art approaches.

AIGC can produce realistic AI-generated images that challenge human perception. Detecting AI-generated content is critical, which has prompted the technology to tell apart real images from the generated ones. However, the existing methods, such as CNND, LGrad, lack interpretability. Unlike traditional image classification, it is crucial to know why the image can be considered as AI-generated. We introduce a novel AI-generated image detector based on genetic programming (GP), prioritizing both interpretability and classification accuracy. This application of GP in this context emphasizes the need for interpretability in AI-generated content identification. Our GP-based approach not only achieves competitive classification accuracy but also provides transparent decision-making processes, bridging the interpretability gap. This method enhances trust and understanding in the AI-generated image detection process. Through extensive experiments, we highlight the potential of GP-based detectors for this unique task. This research contributes to improving the transparency and reliability of AI-generated image detection, holding implications for computer vision and image forensics. Our work emphasizes the pivotal role of interpretability in distinguishing AI-generated content and offers insights into the inner workings of such models and also achieves a good generation ability.

Personalized tag recommender systems automatically recommend users a set of tags used to annotate items according to users’ past tagging information. Learning the representations of involved entities (i.e. users, items and tags) and capturing the complex relationships among them are crucial for personalized tag recommender systems. However, few studies have been conducted to simultaneously achieve these two sub-goals. In this research, we propose a novel personalized tag recommendation model based on the denoising auto-encoder, namely DAE-PTR, which learns the representations of entities and encodes the complex relationships by exploiting the denoising auto-encoder framework. Specifically, for each user, we firstly generate the corrupted version of the respective tagging information by adding the multiplicative mask-out/drop-out noise into the original input. Then, we learn the latent representations from the corrupted input via the auto-encoder framework by using the cross-entropy loss. More importantly, we integrate the latent user and item embeddings into the processing of encoding, which makes the learnt hidden representations of the auto-encoder network encode multiple types of relationships among entities, i.e. the relationships between users and tags, between items and tags, and among tags. Finally, we employ the decoder component to reconstruct the original input based on the learnt latent representations. Experimental results on the real-world datasets show that our proposed DAE-PTR model is superior to the traditional personalized tag recommendation models.

Chinese Spelling Check (CSC) aims to detect and correct Chinese spelling errors. Most Chinese spelling errors are the misuse of semantically, phonetically or graphically similar characters. Previous state-of-the-art works on the CSC task pursue transitions from misspelled sentences to correct sentences directly. However, it is difficult to force the current CSC methods to find the correct answer at one run. Thus, we propose a simple and effective method for CSC task by making fully use of the trained model to generate multiple candidate sentences and simply ranking to select the best, in which no additional training and parameters are required. The experimental results show that our approach outperforms previous methods and achieves the state-of-the-art performances.