Machining feature recognition plays a crucial role in the effective integration of Computer-Aided Design (CAD), Computer-Aided Process Planning (CAPP), and Computer-Aided Manufacturing (CAM) systems. However, current machining feature datasets lack diversity and authenticity in both machining features and base models. Existing recognition methods are limited in capturing the geometric information of long-range complex intersecting features. In this paper, we first construct a novel Multi-Machining Feature model Dataset (MMFD) that includes various non-machining feature shapes as base models and complex machining features commonly encountered in real industrial manufacturing processes. We also propose CRNet, a contrastive learning-based machining feature recognition network with a Transformer encoder and Face Geometric Position Encoding (FGPE) to capture long-range contextual and spatial geometric relationship information between complex features. Our experimental results demonstrate the effectiveness and accuracy of our CRNet on MMFD compared to the state-of-the-art methods.
Seamlessly integrating virtual objects into real scenes is a critical challenge in augmented reality. To achieve this, it is important to provide globally consistent estimations of lighting conditions in the real scene. In this study, we propose a method to reproduce the appearance of metallic materials using an image-to-image translation network incorporating Vision Transformer (ViT). The network receives an image consisting of a background and a normal map of a virtual object, and transforms the normal map into a virtual object with the metallic material appearance. Specifically, ViT, which effectively captures long-range dependencies in the input image, is introduced into the network’s encoder to extract lighting information, and the network produces the natural appearance of the virtual object that reflects the extracted lighting information in the output image. We created a synthetic dataset and compared images generated by the proposed method with those generated by a CNN-based image-to-image translation network. The results showed that the proposed method outperformed the CNN-based method in three quantitative metrics and reproduced a more natural appearance of metallic materials that is consistent with the lighting conditions.
Video anomaly detection is crucial for public safety and traffic management but faces challenges due to the rarity, context - dependence, and high - dimensionality of anomalies. Traditional fully - supervised methods rely on expensive frame - level annotations. Existing weakly - supervised approaches, dependent on single - modality or static clip modeling, struggle to balance semantic discrimination and temporal coherence. To address these, we propose a novel weakly - supervised video anomaly detection framework using only video - level labels. It employs a Temporal Dynamic Modeling (TDM) module for capturing long - and short - term dependencies, a Semantic - Assisted Anomaly Recognition (SAAR) module to enhance anomaly discrimination, and a Time Series Confidence Smoothing (TCS) strategy to optimize result stability. Our method achieved an AUC of 97.64
While Convolutional Neural Networks (CNNs) excel at local feature modeling, they struggle with global context in lightweight super-resolution (SR) tasks and suffer from high computational complexity. Vision Transformers (ViTs) are effective at capturing long-range dependencies but tend to be computationally expensive and less efficient at handling local details. To address these challenges, we propose Hybrid-SANet, a novel hybrid self-attention network architecture that combines the strengths of both CNNs and ViTs. To this end, we design a Local Self-Attention (LSA) module consisting of wavelet transform and variable large kernel block to efficiently extract high-frequency details and capture short-range dependencies within the image. This allows the model to effectively preserve fine textures and local features, which are crucial for high-quality SR reconstruction. To enhance the global feature extraction, Hybrid-SANet also incorporates frequency projection into sparse GSA, which enables the model to capture long-range dependencies and global context. Experimental results demonstrate that Hybrid-SANet outperforms existing SR methods on benchmark datasets, particularly in terms of detail recovery and computational efficiency.
In the field of 3D generation, producing high quality physically based material maps has long been a key research challenge. However, mainstream diffusion-based material generation methods often lack fine-grained details and logical consistency. In contrast, manual material creation typically relies on extensive material layering and grouping to enrich details and ensure logical coherence. In this paper, we propose MatLayerNet, a multi-agent-based method that organizes large language model (LLM) agents to simulate the human workflow of material creation for material generation tasks. Firstly, we construct an iterative material segmentation pipeline that decomposes textual descriptions of objects into multiple material texts. Secondly, we deconstruct the logic of material layering, guiding the LLM to generate structured layer plans based on substrate material, material texture, and material aging. Thirdly, we introduce a vertically structured multi-agent architecture to generate consistent and coherent material parameters across all layers. Finally, we incorporate a retrieval-augmented generation (RAG) approach, where the LLM invokes a material mask generator to produce layer-specific masks and determine spatial distribution. Experimental results demonstrate that our method can generate more accurate material details while maintaining consistency and logical integrity among all material components.
Nvshu is a unique Chinese writing system and a gender-specific script in the world. Its characters feature rich variations in stroke direction, numerous intersecting and decorative strokes, and specific writing rules. These characteristics often lead to inaccurate recognition and poor robustness when processed by traditional methods. To address these challenges, this paper proposes a new Nvshu recognition model, Polarity-aware Linear Attention and Learnable Local Salient Kernel (PALLS), which incorporates collaborative optimization of multiple modules in its network architecture. The PALLS model enhances feature extraction by introducing a Learnable Local Salient Kernel Module (LLSKM) after traditional convolution. This module performs multi-scale feature extraction, significantly improving the model’s ability to capture the edges and texture details of Nvshu strokes. Additionally, the model replaces traditional linear attention with a Polarity-aware Linear Attention (PLAttention) module. By decomposing and interactively modeling positive and negative directional features, this module enhances the model’s discriminative power in identifying stroke directions. Furthermore, a Multi-scale Feature Fusion Module (MSFFM) is constructed to integrate semantic information from different levels, enhancing the model’s adaptability to diverse samples. Experiments prove that the PALLS model achieves a recognition accuracy of 92.2
Reconstructing 3D neurons from large-scale electron microscopy images is crucial for brain connectomics. Existing 3D neuron reconstruction pipelines typically rely on labor-intensive manual proofreading to merge neuron fragments generated by automated image segmentation algorithms. To facilitate the proofreading process, we propose a novel neuron tracing framework based on point cloud representations. We formulate neuron reconstruction as a binary classification task that predicts whether a pair of segments should be merged into the same neuron based on their 3D morphology. Inspired by selective state space models (SSMs) that offer sequence modeling capabilities similar to transformers but with linear time complexity, we propose MambaTracingNet to integrate the growth direction of neuron segments into the dependency learning for neuron tracing. Our MambaTracingNet provides robust long-range dependency for neuron segments while maintaining superior computational efficiency based on the selective SSM architecture. We conduct comprehensive experiments on 20,000 segment pairs from the FlyTracing dataset. The results demonstrate that our method outperforms previous point cloud networks with a more lightweight network and significantly reduces GPU memory usage during training.
We propose a weakly supervised framework for classroom teacher tracking based on point annotations. The training data consists of a small portion of fully labeled samples and a large portion of weakly labeled ones. This study aims to train a point-to-box regressor using sparsely fully annotated data. Specifically, we introduce an effective method that utilizes the ViT architecture to predict bounding boxes based on single-point annotations. Our approach generates plausible pseudo-labels for images that only have point annotations. Dataset construction for object tracking is often labor-intensive, and practical deployment requires additional fine-tuning of the model. By leveraging low-cost point annotations, our approach improves label efficiency and reduces annotation costs for practical deployment. We conducted experiments on a classroom teacher dataset, and the results validate the effectiveness of our method. The proposed method establishes a general dynamic modeling framework for weakly supervised single-object tracking, achieving computational efficiency and cross-scenario generalization capability.
Cartoon animation often uses shading to enhance visual appeal and convey 3D effects. However, shading complicates tasks such as segmentation, structural line extraction, and region tracking. Existing shading removal methods for natural videos are not directly applicable to cartoon animations due to the unique characteristics of cartoons. In this paper, we propose a novel network architecture tailored for shading removal in cartoon animations, incorporating attention mechanisms and feature matching loss to improve accuracy while preserving details. To support supervised training, we construct a high-quality dataset of 2,000 manually annotated cartoon animation frames and introduce an automated method to generate additional training samples. Experiments show that our method delivers better visual and quantitative results, and effectively supports tasks like segmentation and region tracking.
In scientific visualization, preserving low resolution volumetric data and employing super-resolution techniques to restore or enhance resolution for further analysis is an effective approach. Currently, neural network methods, particularly those based on implicit neural representations (INR), are widely used for this purpose. However, serious overfitting often arises when INR networks are used in super-resolution tasks, significantly impacting visualization results. This problem has not been previously discussed or resolved. Therefore, our aim is to address the challenges faced by INR networks in super-resolution tasks, pursuing improved reconstruction results and a more stable training process. We propose an approach based on a priori and a posteriori constraints that enhances super-resolution performance with minimal computational overhead. The core idea of this approach is to introduce constraints with additional information to bridge the gap between the model’s optimization goal and the specific objectives of the super-resolution task, thereby mitigating overfitting. We conducted multiple experiments on ensemble simulation datasets covering domains such as the universe, turbulence, and ignition. Comparisons with several state-of-the-art methods demonstrate the effectiveness of our approach.
In order to overcome the high computational complexity, existing ultra-high-definition (UHD) image deraining methods usually leverage downsampling operations in the spatial domain, such that UHD deraining becomes possible on resource-limited devices. However, directly performing downsampling in the spatial domain easily leads to information loss, which further increases the difficulty of image deraining. To address this issue, we design an effective wavelet neural network (EWNet) for UHD image draining. The EWNet introduces wavelet transform to alleviate the information loss of image downsampling on the one hand, and moreover, it decomposes the high-frequency and low-frequency information of the image to remove rain streaks at different levels respectively. To retain the expected texture information of the target image when removing the rain information of the low-frequency image, we design a low-frequency information preservation module (LIPM) in EWNet. Furthermore, we propose a high-frequency detail enhancement module (HDEM), which not only takes the decomposed high-frequency information as input but also uses the output information of LIPM to assist in generating the clean high-frequency details. We conducted experiments on publicly available datasets, and the results showed that when with fewer than 1M parameters, our proposed method achieves superior rain removal effects.
Volumetric medical image segmentation has emerged as a fundamental component of modern diagnostic imaging and treatment planning, where it also plays a crucial role in the accurate diagnosis of 3D brain tumors. Convolutional neural networks have dominated in this task for local feature extraction, but their limited receptive fields hinder the capture of long-range dependencies in 3D brain MRI scans. Although Transformer-based approaches are effective at modeling global relationships in input data, their high computational cost poses challenges for scalability and clinical deployment. Inspired by the recent advances in the application of Mamba state space modeling in the area of medical imaging, we propose LA-mUNet an efficient brain tumor segmentation method using: 1) the hybrid Mamba-CNN encoder to seamlessly extract local and global features from all relevant tumor tissues, 2) the Decoder based on Lesion Attention Module (LAM) and efficient Depthwise Separable Convolutions (DSC) to capture fine-grained details of small tumor lesions. In our lightweight model design with Mamba-LAM integration, we have ensured the balance in performance and computational cost. The proposed method is evaluated on the public BraTS 2023 and BraTS-Africa datasets. The experimental results show the proposed method achieves an average dice score of 91.71
In the task of shuttlecock tracking and strike counting, existing methods suffer from significant limitations in detection accuracy and tracking stability due to the small size and high speed of the target. Furthermore, the lack of a dedicated shuttlecock dataset further restricts the improvement of model performance. To address these issues, this paper proposes a novel network model that integrates a small object detection branch and a convolutional attention mechanism to enhance the detection and tracking capabilities of shuttlecock targets. Specifically, a small object detection branch is added to the detection framework to strengthen high-resolution feature extraction, and a convolutional block attention module is introduced to enhance the representation of key features under complex backgrounds and scale variations. Meanwhile, a dedicated shuttlecock dataset is constructed through real-scene recording and rigorous filtering, effectively alleviating the shortage of training data. Experimental results demonstrate that the proposed method achieves significant improvements in detection precision, recall, and overall mAP metrics, with the overall mAP increasing by approximately 6
3D virtual environments can be alternative domains for gathering synthetic images to train a computer vision (CV) model. While 3D virtual environments could potentially generate unlimited data, manually creating 3D objects and setting up the scenes would still be time-consuming and challenging. Furthermore, real-world constraints must be incorporated into the 3D scene to make them consistent with real-world scenarios. One promising approach is to explore grammar-based procedural content generation (PCG) for generating diverse 3D scenes with real-world constraints represented as a set of grammar rules. In this study, we explored the use of an L-grammar PCG for generating 3D city scenes. Using the generated city scene (DLSU-PCG), we gather synthetic images for training a DeepLabV3+ semantic segmentation network (SSN) and observe if the SSN could perform well on real-world images, such as in the Cityscapes dataset. In addition to our proposed PCG method, we propose a new approach for bridging the synthetic-real domain gap where instead of using domain adaptation (DA) or image-to-image style transfer like other works, we propose manipulating the atlas textures directly by sampling image patches from the Cityscapes dataset. Thus, the synthetic images we gather from DLSU-PCG no longer need a separate DA operation, potentially saving time and cost when generating training data. Quantitative results demonstrate the viability of our approach where our SSN, trained purely on DLSU-PCG images, achieves an almost 5% increase in segmentation accuracy.
Facial action unit (AU) detection is a challenging task, as AUs are subtle, dynamic, and diverse. Recently, the prevailing techniques of visual foundation models (VFMs) and large model fine-tuning have been introduced to many computer vision tasks. However, most existing AU detection methods neglect the fine-tuning of VFMs, and thus still suffer from the difficulty of learning powerful feature representations. In this paper, we propose a new iterative rank reduction adapter (IR ^2 A) to fine-tune a VFM for AU detection. In particular, we freeze the pre-trained model parameters and introduce trainable rank decomposition matrices to top self-attention blocks. We set an initial rank for rank decomposition matrices, and then iteratively reduce the rank via principal component analysis. Moreover, we propose a directional attention to learn relevant features to each AU, in which important information in different directions are captured. Extensive experiments show that our method outperforms state-of-the-art AU detection approaches on challenging benchmarks including BP4D, DISFA, and GFT.
Large-scale point cloud registration is crucial for 3D scene modeling, mapping, and understanding. However, even the most advanced state-of-the-art approaches continue to grapple with the challenge of building correct 3D correspondences and accurately estimating rigid transformations for large-scale point clouds with massive low geometric salience points. In this paper, we propose a novel geometric salience-aware framework for large-scale point cloud registration. First, we present a simple but efficient approach to compute geometric salience priors for both source and target point clouds to classify all points into high-, low-, and non-salience points. Then, we leverage a salience-aware voting method for ranking and selecting correspondences from the initial correspondence set with pairwise compatibility constraints. Finally, with the confidence values of the selected correspondences, we further utilize a robust estimation function to compute the rigid transformation information. Extensive experiments on several benchmarks unequivocally demonstrate substantial enhancements to the state-of-the-art performance in both quality and quantity.
Traditional collaborative modeling design and project file management platforms typically rely on centralized system architectures, which can lead to single points of failure, resulting in data manipulation, project delays, or even complete project restarts due to irreversible data loss. Additionally, the intellectual property of designers is not effectively protected, posing risks of infringement and piracy. Blockchain and zero-knowledge proof technologies can address these issues by ensuring data immutability and privacy protection. In this paper, we propose a privacy-preserving collaborative 3D design framework for copyright protection, which can authenticate and verify ownership without revealing the information about the works. This framework utilizes blockchain to track the revision history by the transaction data model and smart contract, and zero-knowledge proof technology to recognize and protect ownership. Finally, we take blender 3D animation as an example to illustrate and evaluate the performance of our framework. The results indicate that our framework is effective in achieving secure 3D animation production, with latency, TPS (transactions per second), QPS (queries per second), storage costs, IPFS upload speed, and zero-knowledge proof generation cost all within acceptable ranges.
Artistic style transfer is a prominent area of research with significant practical applications. However, existing methods often struggle to achieve a harmonious blend between the content of an image and the style of another, resulting in noticeable artifacts or loss of content fidelity. To address these challenges, we present a novel framework called the Dynamic Style-Adaptive Image Transformation Network, designed to improve the quality of stylized images. We introduce Dynamic Style Module (DSM) that refines style adaptation by dynamically adjusting style features. Additionally, the Attention-Guided Region Styling (AGRS) used to enable precise stylization by focusing on specific image regions, while Progressive Style Blending (PSB) ensures smooth transitions between different style elements. Finally, the Refinement Module polishes the final output for enhanced coherence and visual appeal. Extensive experiments show that the proposed method produces high-quality stylized images while significantly reducing artifacts compared to existing state-of-the-art approaches.
To increase the realism of fluid simulations, interactions between different fluids (such as water and air, or water and oil) must be considered. This allows for the simulation of phenomena such as the ‘glugging’ effect seen when a filled bottle is turned upside-down and air must fill the space left by the liquid. A common approach to fluid animation is to use Particle-in-Cell (PIC) methods. However, such methods are known to lose fluid volume over time due to accumulating numerical errors, an issue that is exacerbated when simulating multiple interacting fluids. Implicit Density Projection (IDP) is used to overcome volume preservation issues for PIC methods. However, it is formulated for single fluids. To address this, we present two novel extensions to IDP: generalised IDP and D1-IDP. Generalised IDP extends IDP to multiple fluids. D1-IDP then further improves on the volume preservation capabilities. We show that D1-IDP is particularly good in multiphase fluid simulations with complex fluid interfaces. We also demonstrate its applicability to multiphase simulations involving variable density fluids. D1-IDP is able to achieve a maximum volume error of <1.0% for the majority of presented scenarios, while having a negligible impact on computation performance compared to generalised IDP. The code used to generate the results presented in this paper can be found at https://github.com/robden820/Multiphase_Fluids .
