Effectively extracting inter-frame motion and appearance information is of great significance for video frame interpolation (VFI). Previous studies either extract these two types of information in a blended manner or develop separate modules for each type of information. These approaches result in ambiguous representations and low efficiency. In this paper, we propose a new family of models, E ^2 MA-VFI, which is equipped with a novel module to explicitly extract motion and appearance information simultaneously. Specifically, we rethink the information processing in inter-frame attention and reuse its attention map for both appearance feature enhancement and motion cue extraction. Moreover, for efficient VFI, the proposed module can be seamlessly integrated into a hybrid architecture combining CNNs and Transformers. This hybrid pipeline can reduce the computational complexity of inter-frame attention while preserving detailed low-level structural information. Meanwhile, to improve parameter-efficiency, we design a lighter and more effective RefineNet network. Compared with the original version, the number of parameters is reduced by more than 70 https://github.com/MCG-NJU/EMA-VFI .
Out-of-distribution (OOD) detection remains a major challenge for deep learning models, especially when test-time OOD data differs substantially from training data. To overcome the static nature of conventional OOD detectors, we propose OODD++, a test-time detection framework featuring a diversified dictionary. Our method introduces a dynamic test-time adaptation mechanism that enables the OOD detector to recognize novel OOD samples upon encountering and improve the detector’s future performance. Its key innovation is a multi-queue and clustering mechanism that prevents the dictionary from becoming outdated or overly specialized, ensuring it captures a wide variety of OOD features. This design is supported by theoretical analysis proving that a more diverse, well-structured dictionary leads to better OOD detection performance. Empirically, OODD++ sets a new state-of-the-art on benchmarks like CIFAR and ImageNet, especially achieving a 12.3
Modern devices commonly deploy multiple cameras with various specifications, such as dual-lens camera systems on smartphones. Because the field-of-views differ between the cameras on smartphones, image resolutions in overlapping regions can vary. Unfortunately, current feed-forward 3D Gaussian Splatting (3DGS) methods are designed only for a set of images with the same resolutions, which limits their performance with the dual-lens images. We observe that existing works struggle to preserve high-frequency details and to account for 3D geometry of scenes from input images with different resolutions and narrow baselines. In this paper, we propose a novel feed-forward 3DGS method that handles dual-lens inputs. To do this, we first consider a dual-lens camera system as a type of cross-scale input. We then adopt a coarse-to-fine feature encoding strategy to effectively enhance extracted low-resolution image features by leveraging geometric information from a depth foundation model. In this process, we improve low-resolution images to provide high-frequency details using a cross-scale enhancement module. Finally, we predict Gaussian parameters through a decoding unit to yield 3D Gaussian primitives. Our model achieves compatible performance in novel view synthesis on public datasets, and state-of-the-art performance on our self-collected dual-lens dataset. We also submit our source codes as supplementary material to validate its robustness.
Camouflaged Object Detection (COD) demands models to expeditiously and accurately distinguish objects which concealed themselves seamlessly in the environment. Owing to the subtle differences and ambiguous boundaries, COD is not only a remarkably challenging task for models but also for human annotators, requiring huge efforts to provide pixel-wise annotations. To alleviate the heavy annotation burden, we propose to fulfill this task with the help of only one point supervision. Specifically, by swiftly clicking on each object, we first adaptively expand the original point-based annotation to a reasonable hint area. Then, to avoid partial localization around discriminative parts, we propose an attention regulator to scatter model attention to the whole object through partially masking labeled regions. Moreover, to solve the unstable feature representation of camouflaged objects with point-based annotation, we perform unsupervised contrastive learning based on differently augmented image pairs. However, the above components mainly emphasize region-level correctness and lack explicit learning of fine boundary structures. To address this issue, we further introduce an Edge-aware Refiner (ER), which evaluates the reliability of pseudo masks based on structural consistency and prediction stability, and performs reliability-guided refinement to enhance boundary discrimination. This design enables the model to learn more precise boundary information and further improves the segmentation quality of camouflaged objects. Extensive experiments demonstrate that our model outperforms several weakly-supervised methods across various metrics.
Estimating the pose of objects from images is a crucial task of 3D scene understanding, and recent approaches have shown promising results on very large benchmarks. However, these methods experience a significant performance drop when dealing with unseen objects. We believe that it results from the limited generalizability of image features. To address this problem, we have an in-depth analysis on the features of diffusion models, e.g. Stable Diffusion, which hold substantial potential for modeling unseen objects. Based on this analysis, we then innovatively introduce these diffusion features for object pose estimation. To verify the efficacy of diffusion features for object pose estimation, we propose three distinct architectures (vanilla, nonlinear, and context-aware weight aggregations) that capture and aggregate diffusion features for comparative analysis. The results demonstrate strong generalization performance in object pose estimation. In real applications, generalizability to unseen data is important, on the other hand, the efficiency is also crucial. To achieve an efficient feature aggregation, we propose a confidence adaptive aggregation network that automatically selects the discriminative features rather than using all the features, achieving a better speed-and-accuracy trade-off. Our method achieves substantial improvements on LM, O-LM, and T-LESS, and also demonstrates measurable gains on the seven core datasets of the BOP Challenge. In particular, our confidence adaptive aggregation network achieves higher accuracy than previous state-of-the-art methods on unseen objects: 97.7 × compared to the CWA variant while maintaining comparable accuracy. Furthermore, CAA reaches real-time performance, achieving over 68 FPS and offering a substantially improved accuracy-efficiency trade-off.
