Instance-level data augmentation methods, exemplified by “copy-paste”, serve as a conventional strategy for improving the performance of small object detectors. The core idea involves leveraging background redundancy by compositing object instances with suitable backgrounds—drawn either from the same image or from different images—to increase both the quantity and diversity of training samples. However, existing methods often struggle with mismatches in background, scale, illumination, and viewpoint between instances and backgrounds. More critically, their predominant reliance on background information, without a joint understanding of instance-background characteristics, results in augmented images lacking visual realism. Empirical studies have demonstrated that such unrealistic images not only fail to improve detection performance but can even be detrimental. To tackle this problem, we propose a scene-understanding-driven approach that systematically addresses these mismatches via joint instance-background understanding. This is achieved through a unified framework that integrates image inpainting, image tagging, open-set object detection, the Segment Anything Model (SAM), and pose estimation to jointly model instance attributes, background semantics, and their interrelationships, thereby abandoning the random operation paradigm of existing methods and synthesizing highly realistic augmented images while preserving data diversity. On the VisDrone dataset, our method improves the mAP@0.5:0.95 and mAP@0.5 of the baseline detector by 1.6% and 2.2%, respectively. Both quantitative gains and qualitative visualizations confirm that the systematic resolution of these mismatches directly translates into significantly higher visual realism and detection performance improvements.
Existing multimodal large language models show promise in action recognition, especially in complex scenes and similar actions. However, they are vulnerable to irrelevant visual background elements and biases stemming from learned language priors, leading to hallucination—inaccurate predictions caused by incorrect associations. To mitigate these challenges, we propose a dual-strategy collaborative framework. First, we design an adaptive dynamic masking strategy that computes frame-level weights using self-attention, with a dynamically determined masking rate via normal distribution sampling, to filter out irrelevant visual content. Second, we combine instruction-based and reverse prompting strategies. Instruction-based prompts clearly define task objectives, guiding the model to focus on essential actions. Reverse prompts question the model’s reasoning process, encouraging continuous evaluation and preventing reliance on outdated language patterns. Extensive experiments demonstrate that our approach significantly mitigates hallucinations arising from both visual distractions and language prior biases, improving the accuracy in action recognition.
Weak supervised temporal action localization is a task with incomplete supervision, aims to localize action instances under video-level action label supervision. Despite significant progress in recent years, there are still issues of context confusion and local localization, mainly due to the inconsistent goals between the classification model and the localization task. Essentially, it is a problem where coarse-grained label information is difficult to ensure alignment between instance-level data and video-level labels. This problem mainly stems from the lack of precise annotation information, which limits the performance of the task. To address this issue, we propose the incorporation of two types of external knowledge: explicit knowledge, such as a knowledge graph, which aids in extracting intricate action details from label semantics, and tacit knowledge acquired from pre-trained models, facilitating the optimal alignment of vision and text through the utilization of rich potential information. In this article, we initially introduced two types of knowledge into the benchmark method framework separately. Subsequently, we aimed to integrate this knowledge effectively to introduce additional information. Our approach is implemented using a modular, plug-and-play design, which allows for the seamless integration of knowledge into various methods, rendering it an efficient and flexible endeavor. In addition, the experimental results indicate that our method improves performance on the THUMOS’14 dataset and two baseline models.
In video analysis, group activity recognition facilitates the detection of rare but critical events, such as classroom disturbances, street conflicts, or novel strategies in sports games, thereby supporting public safety and strategic decision-making. However, such behaviors occur infrequently in real-world scenarios, resulting in limited training samples and poor generalization of existing models when encountering unseen activity categories under zero-shot conditions. To address this issue, this study proposes a weakly supervised group activity recognition method that requires only video-level labels without frame-wise or individual annotations. The proposed approach learns dynamic representations of group behaviors while integrating textual encoding and a prompt expansion module to achieve joint modeling of visual and semantic features, thus enhancing zero-shot recognition performance. Experimental results on the Volleyball and NBA datasets demonstrate that the proposed model outperforms existing methods in both recognition accuracy and generalization capability, and ablation studies further validate the effectiveness of each module.
Existing studies have achieved automatic coding of classroom dialogue and the mining of learning patterns based on the results of automatic coding. However, they have failed to focus on conducting automatic coding research and data mining analysis on classroom dialogue in native language education. In view of the technological advantages of large language models, this study proposes a strategy that combines the construction of training data and model fine-tuning to achieve automatic coding of classroom dialogue in the field of Chinese native language education. Through large-scale coding, data analysis is carried out to extract learning patterns, and then educational experiments are conducted to verify the effectiveness of these learning patterns. The empirical results show that this study has improved the efficiency of automatic coding of classroom dialogue in Chinese native language education, achieved in-depth mining of subject teaching patterns, and effectively contributed to the improvement of learning outcomes in the classroom.
Flow matching based generative models typically describe trajectories through velocity fields, where the evolution of each trajectory is determined by the instantaneous velocity. While recent single-step generation approaches achieve efficient sampling by modeling the average velocity field, their global uniform-velocity constraint limits generalization to complex generation tasks. To mitigate this limitation, we propose Quadratic Flow, a generative method that dynamically models sampling trajectories through average velocity fields under the assumption of constant acceleration. Specifically, quadratic trajectories are defined for points along each path by establishing a quadratic relationship based on the two endpoints of the trajectory, and the network is trained to approximate the corresponding average velocity fields. This strategy allows different initial velocity fields to induce distinct accelerations, implicitly enforcing a second-order constraint without requiring explicit acceleration supervision, while retaining the training efficiency characteristic of velocity-based approaches. Experimental results on the ImageNet dataset validate that our method achieves superior performance compared to existing approaches.
The core of scene graph generation (SGG) is to understand the relation between objects. The representation of the relation in the image involves the intrinsic properties of the objects and the states in which the objects are involved. The "object semantic" representation determines the recognition of the object label. The "states" representation has a greater influence on the inference of inter-object relations. In a word, there is a fusion in the representation between the object's semantic feature and the relation classification feature. In this paper, we propose a feature disentangle method to enhance the performance of unbiased scene graph generation. By analyzing different causal paths in the structural causal model (SCM), we propose a counterfactual-based mutual information minimization method for feature disentanglement. We dynamically construct the object class centers by using the von Mises-Fisher (vMF) distribution and minimize the mutual information between object and relation features through a counterfactual thinking process that subtracts the object center from the relation feature. The counterfactual thinking process is integrated into the model training process, ensuring that feature representations adhere to a mutual information minimization constraint. The proposed feature disentanglement method enhances the alignment of the structural causal model representation with the actual scene graph generation model training process. Experimental results demonstrate that by using the feature disentanglement method, the scene graph generation model can further improve the debiasing performance.
Generating high-quality, interconnected questions remains a significant challenge in artificial intelligence (AI), particularly in applications requiring logical coherence and social relevance. Current methods often lack a cognitive foundation to ensure meaningful question relationships, limiting their effectiveness in dynamic environments. To address this gap, we propose a novel neuroscience-inspired framework, HiP-QCG (Hippocampal-Prefrontal Question Chain Generation), that integrates AI with theories of prefrontal cortex function, hippocampal memory retrieval, and the dynamic interplay between the Default Mode Network (DMN) and Central Executive Network (CEN). A key feature of this method is its flexible architecture, which can leverage a knowledge base when available via a Retrieval-Augmented Generation (RAG) pathway, but also operates effectively without one through a direct generative pathway, ensuring adaptability in dynamic or niche domains. Our methodology consists of three key steps: (1) the Prefrontal Cortex Simulator, where dual models emulate the dorsolateral prefrontal cortex (DLPFC) for assessing logical complexity and the ventromedial prefrontal cortex (VMPFC) for evaluating social-contextual relevance to categorize questions and scaffold their thematic chunks; (2) the Hippocampus Simulator, which, mirroring human memory, provides thematic cues and routes processing based on task complexity; (3) the DMN/CEN Simulator, where difficulty-based routing refines questions through either associative (DMN) or rigorous (CEN) processing. Experiments show our HiP-QCG method outperforms baselines in coherence, diversity, and human evaluation. This work integrates AI and cognitive science, enabling applications in education and conversational AI. Future work will explore additional cognitive mechanisms.
With recent advancements in generative models, controllable human image generation has achieved significant progress. However, existing methods still struggle in complex scenes involving multiple people and intricate poses, often leading to pose ambiguity and body part mismatches. Consequently, the generated images tend to lack structural coherence and visual realism. To address these challenges, we propose MultiHuman, a novel framework for multi-person human image generation that goes beyond simple pose and interaction modeling. Our approach utilizes coarse human poses and dense depth maps as layout guidance, explicitly tackling the issues of modality alignment and feature fusion in multi-human generation. To this end, we introduce a Modality Alignment Module (MAM) that projects pose and depth representations into a shared embedding space. Spatial alignment is achieved through contrastive learning, which ensures consistent and complementary layout representations across multiple people. In addition, a Pose Modulation Module (PMM) is designed to convert hard joint masks into differentiable soft masks, which improves the precision of pose control. To support training and evaluation under challenging conditions, we construct a new dataset, MTHuman, comprising 110K images that feature multi-person occlusions and complex human poses. Experiments on LAION-Human, HumanArt, and MTHuman show that MultiHuman outperforms existing methods like StablePose, with a 2.36% improvement in controllable human image generation accuracy. MultiHuman shows better image quality and stronger structure control.
Hallucination poses a challenge to the deployment of large vision-language models in applications. Visual information intervention, as an effective approach for mitigating hallucinations, steers model behavior in the intended direction by enhancing the stability of visual feature representations during inference. However, existing visual information intervention methods typically rely on globally steered single-scale representations and lack local multi-scale visual information. This limitation undermines their ability to mitigate hallucinations caused by representational biases across multi-scales. Therefore, we propose a training-free visual information intervention method based on adaptive fusion of multi-scale visual information. First, we construct a multi-scale pyramid structure to capture visual information at different local scales. Then, an adaptive cosine distance weighted aggregation module is designed to dynamically adjust the steering weights of each scale based on the semantic correlation of visual information across different scales, thereby enabling more accurate retention and fusion of multi-scale visual semantic information. Finally, we leverage the activations from intermediate layers to facilitate semantic decoding, thus alleviating the issue where semantically relevant tokens exhibit peak activations in intermediate layers but fail to manifest in the final output layer. Extensive experiments show that the proposed method can effectively reduce hallucinations and outperform state-of-the-art methods on multiple metrics.
Within the framework of United Nations Sustainable Development Goal 4 (SDG 4: Quality Education), adult aesthetic education faces challenges related to the sustainability of access, personalisation, and humanistic values. This study examines levels of acceptance regarding generative AI-assisted aesthetic education in the Open University of China. Drawing on the Technology Acceptance Model (TAM) and lifelong learning theory, we surveyed 518 adult learners and used descriptive analysis (based on the full sample) alongside binary logistic regression and PLS-SEM (based on the 355 learners who had prior aesthetic education experience) to analyse adoption readiness, functional preferences, and influencing factors. Our main findings are as follows: (1) 91.5% of learners recognised the importance of aesthetic education, yet the actual participation rate was only 68.5%; (2) 50% of respondents expressed clear willingness to use AI-assisted aesthetic learning, with the most desired functions being “personalised art interpretation” and “creative inspiration support”; (3) no significant age-related differences were found in AI adoption readiness; (4) logistic regression identified aesthetic cognition and learning experience as significant predictors of AI adoption willingness; and (5) SEM showed that learning experience had the strongest direct effect on AI adoption readiness (β = 0.62) and partially mediated the relationship between aesthetic cognition and AI adoption readiness. We conclude that generative AI should be positioned not as a “substitute” but as a “personalised interpreter and creative partner” in adult aesthetic education. These findings provide practical implications for the digital transformation of The Open University of China and the design of AI-based educational products.
Multimodal large language models (MLLMs) have made strong progress in video understanding, yet understanding long videos remains difficult because of limited contextwindows, temporal redundancy, and high computational cost. Most training-free methods rely on isolated frame selection or fixed-length clip partitioning, which can break event continuity and weaken the alignment between the question and the visual content. We propose EBSLS, an Event-Based Similarity-Length Sampling framework for long-video understanding. EBSLS first segments a long video at a low frame rate into semantically coherent event clips. It then rewrites the input question into auxiliary scene descriptions to improve video-text retrieval and identify question-relevant clips more accurately. Given a limited frame budget, EBSLS allocates frames by considering both clip relevance and temporal length, and then densely samples frames from the selected intervals in the original video for downstream reasoning. Experiments on three long-video understanding benchmarks, MLVU, LongVideoBench, and VideoMME, show that EBSLS is effective. Compared with the plain LLaVA-Video baseline, EBSLS improves performance by 3.73% on MLVU, 2.61% on LongVideoBench, and 1.21% on Video-MME. On MLVU, it achieves the best result among all compared methods, with an M-Avg of 74.52%.
Effective procedure planning in instructional videos requires robust modeling of dynamic step sequences that adapt to contextual variations and diverse execution styles. Current single-modality approaches exhibit complementary limitations: text-based methods struggle to specify spatial relationships and quantitative action parameters, while video-based systems fail to establish task-level semantics and logical step dependencies. To address these challenges, we propose TeViJP, a dual modality framework that integrates textual abstraction with visual demonstration for adaptive workflow planning. Three key innovations in our work have been introduced: 1) A step-semantic aware prompting mechanism using contrastively aligned tokens to dynamically segment procedural phases in long videos, resolving boundary ambiguity under variable step orders; 2) A goal-conditioned hierarchical planner that injects multi scale task semantics through cross-modal attention, reducing error accumulation in long-horizon planning; 3) A Temporal Spatial-Semantic Fusion Transformer (TSSB) ensuring synthe sized previews maintain geometric consistency with historical states while aligning with textual constraints. Comprehensive evaluations across four benchmark datasets demonstrate superior performance, achieving 4.63% accuracy gains in step prediction and halving visual distortion metrics compared to state-of-the-art planning systems.
Classroom behaviour recognition is easily disrupted by visual confounders such as dense crowds, occlusions, viewpoint and illumination changes, and closely co-occurring actions. These factors hinder the learning of robust and interpretable behaviour representations. Causal inference has been introduced into this task, but existing methods typically treat spatio-temporal confounders as a single factor and ignore the distinct roles of spatial and temporal information. This contrasts with evidence from cognitive science that “where” and “when” information is processed by partly separate neural systems. Existing classroom behaviour datasets further focus on isolated actions and rarely capture the dynamics of interaction. To address these issues, we propose CausalCIBR, a causal-inference-based framework for classroom interactive behaviour recognition, together with a new dataset, BNU-SVIBD. CausalCIBR models spatial and temporal behaviour associations as decoupled confounders, whose occurrence probabilities are calibrated using behaviour co-occurrence statistics from educational practice. Within a structural causal model, we intervene on these confounder dictionaries and use a multi-scale gated feedforward module to couple intervened features with local and global spatio-temporal context, strengthening causal signals while suppressing spurious co-occurrences. BNU-SVIBD provides annotated interactive sequences that describe the dynamic evolution of classroom interaction events, enabling fine-grained behaviour analysis. Experiments on BNU-SVIBD and other challenging benchmarks show that CausalCIBR achieves state-of-the-art performance and improved robustness under distribution shifts and confounder perturbations, effectively capturing the complexity of classroom interactions.
Weakly-Supervised Temporal Action Localization (WS-TAL) aims to localize actions in untrimmed videos using only video-level labels. The core challenge is the lack of fine-grained annotations, which leads to high prediction uncertainty and confusion between actions and background. To address this, we propose an Uncertainty-Aware and Knowledge-Selection (UAKS) approach. Specifically, we integrate two uncertainty estimation strategies to cooperatively optimize the model and leverage uncertainty to guide external knowledge selection. First, evidential learning estimates model uncertainty, generating more confident predictions via regularization. Second, probabilistic distribution learning captures data uncertainty. Both uncertainties jointly guide model optimization. Additionally, uncertainty-driven knowledge selection enables the efficient utilization of external knowledge under weak supervision. Experiments show that our method improves accuracy and robustness, with 12.9% and 2% accuracy improvements on THUMOS and ActivityNet v1.3 datasets respectively, demonstrating the potential of uncertainty modeling in WS-TAL.
Few-Shot Open-set Recognition (FSOR) is a challenging task as it poses the dual challenge of classifying known classes with limited labeled samples while effectively identifying unknown classes. Existing approaches typically rely solely on visual features to construct negative prototypes for modeling the open-set space, which limits their representational capacity and generalization. These methods overlook the rich semantic information in class names, which can significantly enhance the modeling of known and unknown spaces. To address this limitation, we propose Multi-Modal Negative Prototypes (MMNP), a novel FSOR framework that integrates visual and semantic modalities to generate discriminative multi-modal class and negative prototypes. MMNP introduces a Semantic Extractor and a Feature Fusion Network to construct discriminative multi-modal class prototypes. In the meta-learning phase, we introduce a Multi-Modal Negative Prototypes (NPs) Generator to generate multi-modal negative prototypes via an attention mechanism. We validate the effectiveness of our method through extensive experiments on two standard FSOR datasets, demonstrating that MMNP outperforms existing state-of-the-art FSOR methods in both 1-shot and 5-shot settings.
Existing scene graph generation methods primarily focus on addressing the long-tail problem in the labeling. However, most debiasing approaches struggle with a trade-off between head and tail class performance compared to biased-trained models. In this paper, we propose a balanced fusion strategy to leverage the strengths of both model types. From the perspective of causality, we proposed a mutual supervision intervention (MSI) method, which consists of a batch-level intervention and a model-level alignment part. This method is integrated into the existing Introd framework to address the bias in the fusion process of scene graph generation models. We propose a new metric called DP@100 to compare the output of the student model and the teacher model, providing an evaluation of the fusion performance under this distillation framework. The results of experiments performed on the Visual Genome dataset show the effectiveness of the proposed MSI method.
The operation fields in industrial control protocols play a critical role in recognizing industrial control network behavior, understanding and monitoring network activities, and accurately identifying and extracting operation fields from industrial control network traffic. However, current methods for operation field recognition often rely on expert experience or manual analysis based on program execution, resulting in low efficiency, limited generalizability, and an inability to handle many undisclosed proprietary protocols or automatically recognize operation fields in complex network scenarios with unknown contexts and protocols. Therefore, this study uses the unique domain characteristics of industrial control networks and proposes an operational field recognition method based on the steady-state properties of industrial control protocols, overcoming the limitations imposed by protocols and programs. First, by preprocessing industrial control network session data, such as session reconstruction and fragmented packet reassembly, the value sequences of various fields at the application layer of the data packets are extracted. Then, through analysis of the stability, periodicity, and correlation of these value sequences, operation fields exhibit steady-state properties characterized by stability, high periodicity, and high correlation. These steady-state properties are quantified as features of operation fields. Next, an unsupervised clustering method is employed to effectively distinguish operation fields from other fields, ultimately achieving automatic recognition of operation fields. The proposed method demonstrates significant value in industrial control protocol security testing, regulating industrial control behavior, and anomaly detection in industrial control systems. For example, by utilizing the recognition results of operation fields, it becomes possible to construct and generate effective fuzzy testing data to enhance the security of industrial control systems. Through extensive validation in various industrial control system environments, including power grids, water treatment experimental platforms, and real industrial control traffic data, the method achieves a recognition rate of over 90% for operation fields, demonstrating its effectiveness and generalizability. In addition, in the experimental section, the influence of data size and quality on the method is discussed in detail. The proposed method accomplishes the recognition task with relatively small amounts of data but requires high-quality traffic data with minimal artificial operations or noise in the industrial control system traffic. Therefore, in practical applications, it is important to ensure the accuracy and purity of industrial control system traffic data. In conclusion, the operation field recognition method based on the steady-state properties of industrial control protocols rapidly and accurately identifies operation fields by analyzing features such as stability, periodicity, and correlation, without relying on specific protocol specifications or source code analysis. The method provides essential technical support for industrial control network security monitoring and behavior analysis, while also providing new possibilities for intelligent control and management of industrial control systems.
Classroom dialogue evaluation is a crucial component of the teaching process assessment. It not only enhances the quality of classroom interaction and student engagement but also fosters the development of students’ core literacy. By leveraging classroom dialogue evaluation, teachers can implement personalized teaching strategies, overcoming the problem of unequal distribution of educational resources caused by regional disparities. However, current classroom dialogue evaluation frameworks are manually implemented, which results in low automation, high time consumption, and a lack of systematic assessment related to core literacy. To address these limitations, we propose the Classroom Dialogue Evaluation Framework for Chinese Core Literacy (CDE-CCL). This framework integrates Chinese Core Literacy, dialogue round segmentation, prompt engineering, and LoRA fine-tuning techniques to encode complete classroom dialogues, thereby enabling automated evaluation of classroom dialogues. Experimental results demonstrate the superiority of CDE-CCL in both tasks of classroom dialogue segmentation and encoding over existing methods across various aspects and educational levels.
High-quality question generation is crucial for ensuring the fairness and validity of examinations. To address the challenges of data scarcity and semantic complexity in automatic question generation (AQG) for niche subjects, including the arts, this study develops a domain-specific large language model (LLM) with a three-tiered optimization mechanism, incorporating prompt tuning, knowledge enhancement, and data augmentation. The model's effectiveness was validated through a case study conducted on a calligraphy course. The results showed that the generated questions achieved a usability rate of 91%, whereas the proposed data augmentation strategy expanded the question bank by 132.56%. This work provides both technical solutions and practical reference for automatic question generation methods targeting niche disciplines. The key contributions of this study encompass the creation of an innovative three-tiered optimization framework, the effective integration of external domain knowledge, and an iterative data augmentation approach that enhances question generation for niche subjects. This research offers a technological pathway and serves as a valuable reference for AQG in niche disciplines.