Event-based Action Recognition (EAR) possesses the advantages of high-temporal resolution capturing and privacy preservation compared with traditional action recognition. Current leading EAR solutions typically follow two regimes: project asynchronous event streams into dense event frames and adopt powerful frame-specific networks, or employ point-specific networks to handle sparse event points directly. However, these two regimes are usually explored in isolation, leaving the complementary properties of event frames and event points insufficiently exploited. In this article, we propose EventPro, a prompt-induced frame and point synergy framework for EAR. Built upon a basic event-frame encoder, EventPro introduces a lightweight yet event-specific point branch to complement frame representations with high-resolution asynchronous motion cues. To bridge the heterogeneous frame and point representations, we further employ prompts to align them in semantic space, thereby producing more discriminative event features. Specifically, the point branch organically integrates two designs: a Spiking-like Context Learner that adaptively extracts contextual event points from raw event streams, and an Event-Point Encoder that models long-range spatiotemporal dependencies among event-point tokens while preserving local topology. Extensive experiments on five benchmarks demonstrate that EventPro achieves highly competitive performance with limited additional computational cost.
Semantic segmentation and disparity estimation are fundamental tasks for visual scene understanding, with critical applications in robotic perception and autonomous driving. Joint learning of both tasks has shown substantial benefits in computational efficiency and overall performance. However, existing multi-task methods often fail to fully exploit the intrinsic semantic-geometric complementarity, leaving substantial room for improving both accuracy and efficiency. We revisit joint semantic segmentation and disparity estimation through semantic-geometric duality, defined as the bidirectional relationship between semantic classes and disparity patterns. Based on region-level and boundary-level observations, we propose a lightweight framework that selectively exploits semantic-geometric duality. Specifically, a Geometry-guided Semantic Modulation (GSM) module injects disparity-derived geometric cues into the semantic branch, while a Semantic-guided Disparity Refinement (SDR) module uses semantic features to selectively refine disparity predictions. In addition, a Boundary Geometry Coupling (BGC) loss introduces class-pair-aware supervision on boundary pixels without increasing inference cost. Experiments on Cityscapes, KITTI 2015, and vKITTI 2 demonstrate that the proposed method achieves strong joint perception performance and a favorable accuracy-efficiency trade-off, using only 8.22M parameters. Real-world deployment on an edge device further demonstrates its real-time inference capability and practical applicability.
Deep learning has achieved strong performance in image recognition, but its decision process remains difficult to interpret. Concept Bottleneck Models (CBMs) make predictions through an intermediate concept bottleneck, exposing the predicted concepts as forward-pass explanations for each recognition result. However, conventional CBMs require concept-level supervision, which limits their scalability. To reduce this cost, recent methods discover concepts from learned representations and use generative models to show concept interventions as image-space changes. These methods reduce manual annotation, but they still need to balance recognition performance, explanation compactness, and generation quality. Strong bottleneck compression or sparsity constraints can remove class-discriminative information, while generated explanations may fail to preserve instance details. In addition, discovered concepts often lack semantic names, making the explanation process rely on extensive manual inspection and limiting practical use. We revisit automatic concept bottlenecks from the perspective of disentangled representation learning and propose the Nonnegative Factorized Semantic Concept Bottleneck Model (NFS-CBM). NFS-CBM formulates the concept bottleneck as a non-negative factorized discriminative representation, where concept units additively decompose the information used for recognition. It then uses a concept-conditioned diffusion model to convert prediction-relevant concepts into visual interventions. After training, NFSCBM uses a vision-language model to summarize cross-sample consistent visual changes and generate semantic descriptions for discovered concepts. Experiments on multiple image classification benchmarks show that NFS-CBM improves both recognition performance and explanation quality. Code is available at https: //github.com/CloudOcean-hue/NFS-CBM.
Completing dense depth maps from 2D visual images and sparse depth measurements is crucial to robustly perceive 3D scenes. Existing end-to-end models have made significant advances under in-domain settings, but do not generalize well to real-world scenarios. One limitation lies in the fact that these models rely heavily on either absolute L1 and L2 relations to learn absolute distance in the same scene, or relative scale-invariant relations to learn relative geometry for cross-scene generalization. However, both paradigms cannot achieve cross-scene generalization in depth completion. It remains a challenge to jointly preserve relative geometry and absolute distance in an end-to-end network. Though recent methods leverage relative geometry priors from pretrained depth foundation models to boost generalization, this challenge was not resolved. To overcome this challenge, we formulate a novel joint absolute-relative relation with three learnable factors, namely, scale, offset, and probability. This relation simultaneously constrains absolute distance and relative geometry of output depth maps with GT depth and sparse depth maps. Based on this relation, we propose a new end-to-end network architecture with two sequential branches, namely, visual and refine, for depth completion. These two branches, respectively, learn the relative geometry and the absolute distance of the scene. The visual branch utilizes a modified U-Net to generate dense depth from sparse ones. The refine branch utilizes another modified U-Net to learn three maps based on the three learnable factors from sparse depth and dense depth. Output dense depth is finally refined with the three learned maps through our joint relation. Our model is trained on mixed indoor and outdoor datasets, and extensively evaluated on ten unseen datasets with a wide range of simulated and raw sparse depth points. Our model outperforms existing end-to-end models on average and even achieves the best accuracy and complexity trade-off compared to recent pretrained methods.
Omnidirectional videos impose intensive demands on efficient coding due to high spatial resolution and unique projection characteristics. This paper investigates spherical rotation angle prediction for omnidirectional video coding and proposes a content-adaptive spherical rotation algorithm that fuses motion and texture characteristics under the joint optimization framework of Equi-Angular Cubemap (EAC) and spherical rotation. Specifically, an adaptive motion-intensity classification scheme is designed to categorize videos into high- and low-motion classes based on average motion energy. For low-motion videos, texture-gradient analysis is utilized to achieve robust and low-complexity rotation angle prediction. For high-motion videos, a motion vector (MV) statistics-based method with a magnitude-weighted directional histogram is designed to accurately estimate the dominant rotation angle. The proposed dual-branch method avoids pre-encoding and maintains low computational complexity. Experimental results demonstrate that the proposed scheme achieves an average bitrate saving of 0.52% and a maximum bitrate saving of 1.89% for specific test sequences compared to EAC. This work validates the effectiveness of joint projection-rotation optimization and provides a practical solution for efficient omnidirectional video coding.
