In the context of the national promotion of intelligent manufacturing and the 'Industry4.0' strategy, the demand for intelligent robotic arms in industrial production has steadily increased. However, challenges such as occlusion, significant object scale variations, and strict real-time requirements have made target detection in densely packed environments more challenging. This study, based on the YOLOv11 algorithm, proposes an efficient oriented bounding box detection method aimed at improving the model's performance in feature extraction, computational efficiency, and network lightweighting to tackle target detection challenges in dense industrial settings. A Dynamic-Cross-Stage-Dual-Conv module was designed to enhance the Bottleneck section, employing a parallel dual-branch structure for local feature extraction and global context fusion. Simultaneously, a CIoU loss function with geometric perception was introduced to improve object localization accuracy and strengthen the network's ability to handle densely stacked objects. Next, a Modulated-Deform-Conv deformable convolution module was integrated into the Backbone structure, dynamically adjusting the convolution kernel sampling positions, enabling the network to learn deformation features in dense scenes, improving adaptability to shape and scale variations while reducing computational load. Additionally, a C3K2_FasterBlock lightweight structure, utilizing partial convolutions and sparse connections, was proposed to minimize redundant calculations and optimize feature interactions. On a custom-built dense object dataset, the model achieved a 2.9% increase in mAP@0.5 and reduced computational cost by 14 %. Finally, the improved model was deployed on the Jetson Orin Nano development board, demonstrating its practical value in robotic arm recognition and grasping tasks in dense industrial environments, offering a new paradigm for future applications.

Vision Mamba (ViM) adapts the Mamba to images via bidirectional state-space scanning, delivering strong accuracy with lower latency and memory footprint. Recent FPGA accelerators largely optimize the original Mamba formulation, with quantization co-design and generic SSM dataflows, rather than ViM’s bidirectional scan setting, which causes difficulties for fully streaming implementations and larger resource overhead. To solve these challenges, we present ViMA, an FPGA accelerator tailored for Vision Mamba. ViMA decouples computation into a Selective SSM module and a Full-stream module with a streaming architecture. At runtime, an interleaved, time-multiplexed schedule exposes bidirectional scan parallelism without duplicating resources. We further introduce a resource-space exploration method that finds balanced configurations and enables deeper pipeline optimizations under a fixed resource budget. Implemented on a Xilinx ZCU15EG platform, ViMA achieves up to 30.2× and 2.84× higher energy efficiency than NVIDIA RTX 3090 and RTX A6000 baselines, respectively, demonstrating that decoupling and resource-space exploration are effective for ViM on edge FPGAs.

The automatic reconstruction of 3D computer-aided design (CAD) models from CAD sketches has recently gained significant attention in the computer vision community. Most existing methods, however, rely on vector CAD sketches and 3D ground truth for supervision, which are often difficult to be obtained in industrial applications and are sensitive to noisy inputs. We propose viewing CAD reconstruction as a specific instance of sparse-view 3D reconstruction to overcome these limitations. While this reformulation offers a promising perspective, existing 3D reconstruction methods typically require natural images and corresponding camera poses as inputs, which introduces two major significant challenges: (1) modality discrepancy between CAD sketches and natural images, and (2) difficulty of accurate camera pose estimation for CAD sketches. To solve these issues, we first transform the CAD sketches into representations resembling natural images and extract corresponding masks. Next, we manually calculate the camera poses for the orthographic views to ensure accurate alignment within the 3D coordinate system. Finally, we employ a customized sparse-view 3D reconstruction method to achieve high-quality reconstructions from aligned orthographic views. By leveraging raster CAD sketches for self-supervision, our approach eliminates the reliance on vector CAD sketches and 3D ground truth. Experiments on the Sub-Fusion360 dataset and several cases on other types of CAD sketches demonstrate that our proposed method significantly outperforms previous approaches in CAD reconstruction performance and exhibits strong robustness to noisy inputs. The source code and all related data will be released upon acceptance of the paper.

Pedestrian detection holds significant practical application value in fields such as urban public safety and intelligent traffic management. However, due to the influences of multi-scale variations, occlusion interferences, and complex environmental factors, pedestrian detection tasks in real-world scenarios still face numerous challenges. To address this, an efficient pedestrian detection architecture, the Bidirectional Fusion and Adaptation Network (BFA-Net), is proposed in this paper. In this method, a Bidirectional Multi-layer Fusion Network (BMFN) is designed to enhance the model's perception capability of multi-scale features. A Hybrid Stride Downsampling (HSD) module is adopted to improve the semantic representation of target features. An Adaptive Multi-scale Slimming Framework (AMSF) is introduced to balance detection accuracy and computational efficiency. In addition, a high-quality pedestrian detection dataset, UrbanCrowd-Ped, is constructed. Experimental results demonstrate that BFA-Net achieves 78.3% mAP@0.5 and 50.5% mAP@0.5:0.95 on the UrbanCrowd-Ped dataset, and 78.6% and 53.2% respectively on the CityPersons dataset, outperforming existing mainstream methods. This fully validates the effectiveness and application potential of the proposed method.

Behavior recognition is a highly challenging task, particularly in scenarios requiring unified recognition across both human and animal subjects. Most existing approaches primarily focus on single-species datasets or rely heavily on prior information such as species labels, positional annotations, or skeletal keypoints, which limits their applicability in real-world scenarios where species labels may be ambiguous or annotations are insufficient. To address these limitations, we propose a query-based Multi-Granularity Behavior Recognition Network that directly mines cross-species shared spatiotemporal behavior patterns from raw video inputs. Specifically, we design a Multi-Granularity Query module to effectively fuse fine-grained and coarse-grained features, thereby enhancing the model's capability in capturing spatiotemporal dynamics at different granularities. Additionally, we introduce a Category Query Decoder that leverages learnable category query vectors to achieve explicit behavior category modeling and mapping. Without relying on any extra annotations, the proposed method achieves unified recognition of multi-species and multi-category behaviors, setting a new state-of-the-art on the Animal Kingdom dataset and demonstrating strong generalization ability on the Charades dataset.

Existing human-object interaction (HOI) motion captioning methods typically describe what happens while referring to the subject using generic terms such as "a person" or "someone", without grounding the caption in subject identity. To address this limitation, we introduce Identity-Aware Human-Object Interaction Motion Captioning task. This task requires each generated caption to specify both the subject identity and the corresponding HOI motion. For example, the model generates "Sub_ID lifts the chair" rather than "A person lifts the chair". For this task, we design identity-aware HOI motion captions based on the BEHAVE and InterCap datasets. We further propose ID-HOINet, which learns from multi-view videos while supporting single-view identity-aware HOI motion caption generation. ID-HOINet contains two core components: Multi-View Identity-Motion Learning Module (MVIML) and Two-Stage Caption Rewriting Strategy (TSCR). MVIML learns from multi-view videos by modeling dependencies across temporal stages and camera viewpoints, capturing identity and interaction motion features. At inference, the TSCR first retrieves the subject identity and generates identity-agnostic HOI motion captions. TSCR then rewrites these captions with the predicted identity to produce the final identity-aware HOI motion captions. Experiments demonstrate that ID-HOINet achieves state-of-the-art performance. Code will be released upon acceptance.

Image-point class incremental learning is a pioneering task that enables expert systems to continuously acquire knowledge from dense 2D visual sources for recognition in sparse 3D physical spaces. To accurately reflect real-world asymmetrical data availability, we define this task as multimodal base-task pretraining plus 2D-only incremental updates. However, existing frameworks cannot address the persistent training-testing modality gap. Thus, we propose a framework that utilizes the randomly masked rendering strategy to establish foundational visual-geometry alignment during pre-training. By learning from semantic-agnostic image-point pairs, it prevents the model from relying on high-level semantic shortcuts and compels it to perceive intrinsic spatial patterns. In the incremental phase, we introduce probabilistic prototype modeling, which functions as a distribution-regularized prototype framework. It leverages a fixed-variance assumption as a structural regularization constraint during incremental training to mitigate the inherent stochasticity of cross-modal mapping. It effectively mitigates the catastrophic forgetting of both categorical knowledge and cross-modal correspondences, while maintaining plasticity for novel classes. Experiments demonstrate that our method achieves state-of-the-art results, outperforming baseline methods by a large margin. The code is available at https://github.com/chaoqi7/CMIP-CIL.

Ensuring the safety of autonomous driving systems against adversarial attacks is a critical challenge, especially when such attacks occur in the physical world. Existing methods typically focus on assessing safety in isolated images, failing to address the continuous operation of autonomous vehicles under adversarial conditions. To overcome this limitation, this study introduces a comprehensive framework that evaluates both physical and virtual safety to ensure sustained vehicle safety during continuous operation. The framework consists of two key innovations: the three-dimensional (3D) physical implementation of adversarial textures and a safety evaluation framework using novel metrics. The 3D physical implementation provides a robust platform to test the system response to physical threats. The second and primary innovation, the safety evaluation, introduces the Safety Score to quantify physical safety and the Collaborative Efficiency Deterioration Rate (CEDR) to assess virtual safety, with the ultimate objective of improving driving safety under adversarial conditions. Simulated experiments validate the effectiveness of these metrics in identifying security vulnerabilities and provide insights into improving the resilience of autonomous driving technologies. Although the evaluation is conducted in simulation, it is designed to reflect real-world constraints. This research provides a reference for the safety evaluation of automatic driving in practical applications.

In respiratory sound classification, conventional self-supervised methods struggle with data scarcity and class imbalance. To address this problem, we propose a dual-branch learning framework that integrates masked reconstruction with contrastive learning. Our method employs an asymmetric encoder-decoder to reconstruct the masked patch and introduces a supervised classification head to extract and reconstruct the original spectral signals, enhancing the understanding of the global semantics of the encoder. Meanwhile, contrastive learning is utilized to strengthen the discriminative learning ability between instances. By jointly optimizing reconstruction, classification, and contrastive objectives, the framework effectively mitigates data limitations and improves generalization. Experiments on the ICBHI 2017 and SPRSound datasets demonstrate superior performance, outperforming existing self-supervised approaches. This validates the effectiveness of combining masked reconstruction with contrastive learning for the classification of respiratory sounds.

Recent advancements in 3D reconstruction technologies have paved the way for high-quality and real-time rendering of complex 3D scenes. Despite these achievements, a notable challenge persists: it is difficult to precisely reconstruct specific objects from large scenes. Current scene reconstruction techniques frequently result in the loss of object detail textures and are unable to reconstruct object portions that are occluded or unseen in views. To address this challenge, we delve into the meticulous 3D reconstruction of specific objects within large scenes and propose a framework termed OMEGAS: Object Mesh Extraction from Large Scenes Guided by GAussian Segmentation. Specifically, we propose a novel 3D target segmentation technique based on 2D Gaussian Splatting, which segments 3D consistent target masks in multi-view scene images and generates a preliminary target model. Moreover, to reconstruct the unseen portions of the target, we propose a novel target replenishment technique driven by large-scale generative diffusion priors. We demonstrate that our method can accurately reconstruct specific targets from large scenes, both quantitatively and qualitatively. Our experiments show that OMEGAS significantly outperforms existing reconstruction methods across various scenarios.

Dense audio-visual event localization (DAVE) aims to identify event categories and locate the temporal boundaries in untrimmed videos. Most studies only employ event-related semantic constraints on the final outputs, lacking cross-modal semantic bridging in intermediate layers. This causes modality semantic gap for further fusion, making it difficult to distinguish between event-related content and irrelevant background content. Moreover, they rarely consider the correlations between events, which limits the model to infer concurrent events among complex scenarios. In this paper, we incorporate multi-stage semantic guidance and multi-event relationship modeling, which respectively enable hierarchical semantic understanding of audio-visual events and adaptive extraction of event dependencies, thereby better focusing on event-related information. Specifically, our eventaware semantic guided network (ESG-Net) includes a early semantics interaction (ESI) module and a mixture of dependency experts (MoDE) module. ESI applys multi-stage semantic guidance to explicitly constrain the model in learning semantic information through multi-modal early fusion and several classification loss functions, ensuring hierarchical understanding of event-related content. MoDE promotes the extraction of multi-event dependencies through multiple serial mixture of experts with adaptive weight allocation. Extensive experiments demonstrate that our method significantly surpasses the state-of-the-art methods, while greatly reducing parameters and computational load. Our code will be released on https://github.com/uchiha99999/ESG-Net.

Weakly supervised Audio-Visual Video Parsing (AVVP) aims to recognize and temporally localize audio, visual, and audio-visual events in videos using only coarse-grained labels. Faced with the challenging task settings, existing research advances along two main paths: pre-training pseudo-label generators for fine-grained cross-modal semantic guidance, or refining AVVP model architectures to enhance audio-visual fusion. However, since audio and visual signals are typically unaligned, achieving accurate video parsing fundamentally relies on precise perception of uni-modal events. Yet these multi-modal focused strategies excessively emphasize multi-modal fusion while inadequately guiding and preserving uni-modal semantics, resulting in noisy pseudo-labels and sub-optimal video parsing performance. This paper proposes a novel framework that enhances uni-modal representations for both the pseudo-label generator and the AVVP model. Specifically, we introduce a similarity-based label migration approach to annotate pre-training data, thereby enabling the pseudo-label generator to better understand uni-modal events. We also employ a soft-constrained manner to refine modeling of uni-modal features in parallel with multi-modal fusion. These designs enable coordinated attention to both uni-modal and cross-modal representations, thus boosting the localization performance for events. Extensive experiments show that our method outperforms state-of-the-art methods in both pseudo-label and AVVP performance.

Previous methods typically rely solely on the image modality to recognize group activity. However, single-modality approaches are susceptible to modality-specific limitations, such as difficulty in distinguishing visually similar group activities by the image modality. This motivates us to explore whether complementary information from other modalities can be extracted to supplement image information. In fact, action labels provide clear text information to express the action's semantics, which existing methods often overlook. Thus, we propose ActivityCLIP, a plug-and-play method that mines the text information contained in action labels to supplement image information, enhancing group activity recognition. ActivityCLIP consists of text and image branches, where the text branch is plugged into the image branch (the off-the-shelf image-based method). The text branch includes Image2Text and relation modeling modules. Specifically, we propose a knowledge transfer module, Image2Text, which adapts image information into text information extracted by CLIP via knowledge distillation. Further, to reduce training complexity, we only add fewer trainable parameters based on the relation module of the image branch to model the interaction relation in the text branch. To show the potential generality of our method, we apply ActivityCLIP to three representative methods, adding only limited trainable parameters but achieving promising performance improvements for each method. We also conduct extensive ablation studies and compare our method with state-of-the-art methods to demonstrate the effectiveness of ActivityCLIP. Code will be available at https://github.com/Xuguoliang12/ActivityCLIP.

Adversarial robustness in multi-modal perception is a critical challenge, particularly as vision systems increasingly rely on explicit 3D representations. Existing adversarial attacks typically optimize perturbations in the 2D pixel domain or focus on single-task degradation, failing to address the joint optimization of consistent 3D parameters that transfer across heterogeneous estimators. In this letter, we present BiTAA, a bitask adversarial attack framework that optimizes 3D Gaussian Splatting (3DGS) parameters to generate physically realizable perturbations. By leveraging differentiable rendering, BiTAA formulates a unified optimization problem capable of simultaneously deceiving object detectors and inducing controllable biases in monocular depth estimation. Specifically, we propose a composite objective function that couples confidence suppression with a signed, magnitude-controlled log-depth bias within regions of interest (ROIs), constrained by geometric and appearance regularization to ensure signal fidelity. Experimental results demonstrate that BiTAA achieves significant cross-task degradation and directional asymmetry in transferability, proving effective in both digital simulation and physical realization scenarios.

Camera-based object detection systems play a vital role in autonomous driving, yet they remain vulnerable to adversarial threats in real-world environments. While existing 2D and 3D physical attacks typically optimize texture, they often struggle to balance physical realism and attack robustness. In this work, we propose 3D Gaussian-based Adversarial Attack (3DGAA), a novel adversarial object generation framework that leverages the full 14-dimensional parameterization of 3D Gaussian Splatting (3DGS) to jointly optimize geometry and appearance in physically realizable ways. Unlike prior works that rely on patches or texture, 3DGAA jointly perturbs both geometric attributes (shape, scale, rotation) and appearance attributes (color, opacity) to produce physically realistic and transferable adversarial objects. We further introduce a physical filtering module to preserve geometric fidelity, and a physical augmentation module to simulate complex physical scenarios, thus enhancing attack generalization under real-world conditions. We evaluate 3DGAA on both virtual benchmarks and physical-world setups using miniature vehicle models. Experimental results show that 3DGAA achieves to reduce the detection mAP from 87.21

Leveraging pre-trained Diffusion Transformers (DiTs) for high-resolution (HR) image synthesis often leads to spatial layout collapse and degraded texture fidelity. Prior work mitigates these issues with complex pipelines that first perform a base-resolution (i.e., training-resolution) denoising process to guide HR generation. We instead explore the intrinsic generative mechanisms of DiTs and propose ResDiT, a training-free method that scales resolution efficiently. We identify the core factor governing spatial layout, position embeddings (PEs), and show that the original PEs encode incorrect positional information when extrapolated to HR, which triggers layout collapse. To address this, we introduce a PE scaling technique that rectifies positional encoding under resolution changes. To further remedy low-fidelity details, we develop a local-enhancement mechanism grounded in base-resolution local attention. We design a patch-level fusion module that aggregates global and local cues, together with a Gaussian-weighted splicing strategy that eliminates grid artifacts. Comprehensive evaluations demonstrate that ResDiT consistently delivers high-fidelity, high-resolution image synthesis and integrates seamlessly with downstream tasks, including spatially controlled generation.

Human-computer interaction (HCI) relies on understanding and adapting to users' emotional states. Micro-expressions (MEs), a critical component of emotional perception, are characterized by their spontaneity, rapidity, subtlety, and difficulty to control. They often reveal an individual's true emotions. A comprehensive and detailed representation of motion is necessary to capture the nuances of facial dynamics effectively. Presently, motion representation methods are predominantly confined to 2D analysis within RGB images, overlooking the critical role of facial structure and its movements in conveying emotions. To overcome this limitation, we introduce an innovative facial motion representation that encompasses 3D facial structure, regionalized RGB and structural motion features. Furthermore, we segment the face into eight distinct regions, selecting only the most significant motion points to delineate the primary motion characteristics of each area. To model the interactions among crucial facial motion regions, we employ an advanced, lightweight point cloud and graph convolution network (Lite-Point-GCN). Comprehensive testing on the CAS(ME)(3 )dataset, using leave-one-subject-out (LOSO), demonstrates that our method outperforms existing state-of-the-art methods.

Vision-and-Language Navigation (VLN) task requires agents to navigate autonomously in 3D visual environments by following natural language instructions. High-quality training data is essential for enhancing both navigation performance and generalization capability. While synthetic path–instruction pairs have significantly increased data scale, template-based generation struggles to align with navigation processes and deviates from real human–robot interaction instructions, leading to an instruction-level sim-to-real gap. To address it, we propose InstructGen, a VLN path-instruction pairs generation framework based on Multimodal Large Language Models (MLLMs), and introduce the corresponding dataset GenVLN. Specifically, we use YouTube house tour videos as realistic navigation scenes and design an innovative prompt engineering strategy to activate MLLM’s visual understanding and text generation capabilities, enabling the model to generate instructions aligned with visual observations and spatial information at varying granularities. Additionally, we design a self-supervised correction mechanism that optimizes the consistency between instructions and navigation trajectories and the fluency of language expressions, reducing potential hallucinations during generation. Experiments on R2R and RxR show that agents trained with GenVLN achieve strong navigation performance and robust generalization in unseen environments, highlighting the effectiveness of high-quality generated instructions.

Traditional human motion prediction (HMP) tasks usually follow a training scheme with the same or similar distributions of action and actor classes.However, it is difficult for training samples to cover all action types and actor characteristics in the real world.This motivates us to propose a novel generalization task for human motion prediction (G-HMP), which uses incomplete action and actor classes during training to predict the new unseen ones.The core objective of G-HMP is to enhance the model's ability to understand action semantics and human kinematic patterns, enabling it to achieve transferable prediction and domain generalization in real-world scenarios.To address the challenges of data scarcity and decreased variety of learnable motion patterns posed by G-HMP, we propose cross-action actor augmentation (CA) and positional blurred motion prediction (PB) methods.CA is a method that enhances dataset diversity by generating new motion sequences across different actions and actors through time reversal, skeletal mirroring, and inertial motion techniques.PB encourages the learning of motion dynamics rather than accurate joint positions by blurring positional information, consequently improving the model's generalization and predictive performance.Experiments on H3.6M and CMU datasets validate the validity of G-HMP and the effectiveness of the proposed methods.

Object detection in dense scenes faces challenges due to high overlap, severe occlusion, and complex backgrounds, which lead to feature confusion. Traditional detectors struggle to achieve stable and discriminative multi-scale representations, resulting in a significant drop in accuracy. To address this scientific issue, a novel architecture optimized for dense scenes was proposed - ELM-YOLO (EMA-LSKA-MSDA-YOLO). This method alleviated the aforementioned problems through three intuitive design innovations: The enhanced C2f-EMA (Faster Implementation of CSP Bottleneck with 2 convolutions - Efficient Multi-Scale Attention) module strengthened key semantics and suppressed background interference using multi-scale efficient attention. The SPPF-LSKA (Spatial Pyramid Pooling Fast - Large Separable Kernel Attention) module expanded the receptive field explicitly with large separable convolution kernels to capture long-range associative features of occluded objects. The MSDA (Multi-Scale Dilated Attention) module achieved more comprehensive multi-scale information fusion with an additional detection head, enabling the model to maintain higher discrimination in dense regions with small objects. Experiments based on a self-built dense scene dataset showed that ELM-YOLO achieved significant improvements over YOLOv8s in accuracy, recall, and mAP@0.5, verifying its effectiveness and applicability in densely occluded environments. This code is available at https://github.com/yunianan866/ELM-YOLO/tree/main .