Equivariant networks embed geometric symmetries as structural priors through weight sharing, achieving remarkable parameter efficiency across vision tasks. However, this parameter efficiency does not translate into compute efficiency: existing implementations unroll the structured weights into dense matrices and dispatch them to generic dense kernels, so the FLOPs of an equivariant layer are no smaller than those of a non-equivariant counterpart. In this paper, we observe that the equivariant linear (EQ-Linear) layer—the most fundamental and frequently used module in modern equivariant architectures—is essentially a circular convolution along the group dimension composed with a linear transform along the channel dimension. Building on this observation, we propose Flash EQ-Linear, an exact acceleration algorithm that reduces the complexity from 𝒪(NDC) to 𝒪(NDC/T) by combining the Fourier convolution theorem along the group dimension with the conjugate symmetry of the real DFT. We further provide dedicated CUDA kernels for Flash EQ-Linear, covering both forward and backward passes and both FP32 and FP16 precision. At the operator level, Flash EQ-Linear achieves up to 2× forward speedup over PyTorch's F.linear; at the network level, Flash EQ-ViT and Flash EQ-Swin achieve up to 1.7× end-to-end speedup over both equivariant and non-equivariant baselines. To our knowledge, this is the first time equivariant networks strictly dominate their non-equivariant counterparts along all three axes simultaneously: accuracy, parameter efficiency, and inference speed.Code is available at https://github.com/zhongchenzhao/FlashEQLinear.

Incorporating symmetry priors as inductive biases to design equivariant Vision Transformers (ViTs) has emerged as a promising avenue for enhancing their performance. However, existing equivariant ViTs often struggle to balance performance with equivariance, primarily due to the challenge of achieving holistic equivariant modifications across the diverse modules in ViTs-particularly in harmonizing the Self-Attention mechanism with Patch Embedding. To address this, we propose a straightforward framework that systematically renders key ViT components, including patch embedding, self-attention, positional encodings, and Down/Up-Sampling, equivariant, thereby constructing ViTs with guaranteed equivariance. The resulting architecture serves as a plug-and-play replacement that is both theoretically grounded and practically versatile, scaling seamlessly even to Swin Transformers. Extensive experiments demonstrate that our equivariant ViTs consistently improve performance and data efficiency across a wide spectrum of vision tasks.

Fourier neural operator (FNO) has demonstrated substantial potential in learning mappings between function spaces, such as numerical partial differential equations (PDEs). However, FNO may suffer from inefficiencies when applied to large-scale, high-dimensional function spaces due to the computational overhead associated with high-dimensional Fourier and convolution operators. In this work, we introduce the Tucker-FNO, an efficient neural operator that decomposes the high-dimensional FNO into a series of 1-dimensional FNOs through Tucker decomposition, thereby significantly reducing computational complexity while maintaining expressiveness. Especially, by using the theoretical tools of functional decomposition in Sobolev space, we rigorously establish the universal approximation theorem of Tucker-FNO. Experiments on high-dimensional numerical PDEs such as Navier-Stokes, Plasticity, and Burger's equations show that Tucker-FNO achieves substantial improvement in execution time and performance over FNO. Moreover, by virtue of the compact Tucker decomposition, Tucker-FNO generalizes seamlessly to high-dimensional visual signals by learning mappings from the positional encoding space to the signal's implicit neural representations (INRs). Under this operator INR framework, Tucker-FNO gains consistent improvements on continuous signal restoration over traditional INR methods in terms of efficiency and accuracy.

In this paper, we present a unified framework for robust 3D embedded watermarking and non-embedded watermarking based on feature integration. It begins by segmenting a 3D model into multiple separate sub-models via empirical mode decomposition (EMD). And then, it constructs a robust feature image for each sub-model by integrating its explicit and implicit radial features. Such scheme enables our framework to seamlessly transition from 3D non-embedded watermarking to 3D embedded watermarking. Our 3D embedded watermarking modifies the model according to its statistical characteristics. Therefore, it is an adaptive embedding method and can improve the invisibility of 3D embedded watermarking. Subsequently, it generates the copyright-watermark keys by using an XOR operation on each feature image and the given watermark image. Additionally, our watermarking framework can extract multiple watermark images according to the feature images of the detected 3D model and the stored copyright-watermark keys. They can be combined into the final watermark via a voting strategy to enhance the robustness of 3D watermarking. The experimental results and analysis demonstrate the superior performance of our newly-proposed 3D watermarking framework in terms of versatility, robustness and invisibility.

Rotation equivariance constitutes one of the most general and crucial structural priors for visual data, yet it remains notably absent from current Mamba-based vision architectures. Despite the success of Mamba in natural language processing and its growing adoption in computer vision, existing visual Mamba models fail to account for rotational symmetry in their design. This omission renders them inherently sensitive to image rotations, thereby constraining their robustness and cross-task generalization. To address this limitation, we propose to incorporate rotation symmetry, a universal and fundamental geometric prior in images, into Mamba-based architectures. Specifically, we introduce EQ-VMamba, the first rotation equivariant visual Mamba architecture for vision tasks. The core components of EQ-VMamba include a carefully designed rotation equivariant cross-scan strategy and group Mamba blocks. Moreover, we provide a rigorous theoretical analysis of the intrinsic equivariance error, demonstrating that the proposed architecture enforces end-to-end rotation equivariance throughout the network. Extensive experiments across multiple benchmarks - including high-level image classification task, mid-level semantic segmentation task, and low-level image super-resolution task - demonstrate that EQ-VMamba achieves superior or competitive performance compared to non-equivariant baselines, while requiring approximately 50

Image restoration is an inherently ill posed inverse problem. Equivariant networks that embed geometric symmetry priors can mitigate this ill posedness and improve performance. However, current understanding of the relationship between network equivariance and data symmetry remains largely heuristic. Particularly for real world data with imperfect symmetry, existing research lacks a systematic theoretical framework to quantify symmetry, select transformation groups, or evaluate model data alignment. To bridge this gap, we conduct an analysis from an optimization perspective and formalize the intrinsic relationship among data symmetry priors, model equivariance, and generalization capability. Specifically, we propose for the first time a quantifiable definition of non strict symmetry at the dataset level (rather than sample level) and use it as a constraint to formulate the restoration inverse problem. We then show that the equivariance for restoration models can be naturally derived from this inverse problems incorporated the proposed symmetry constraints, and that the equivariance error of the optimal restoration operator is strictly bounded by the data symmetry error and the discretization mesh size. Furthermore, by analyzing the network's empirical risk, we demonstrate that aligning equivariance with data symmetry optimizes the bias variance trade off, minimizing the total expected risk. Guided by these insights, we propose a Sample Adaptive Equivariant Network that uses a hypernetwork and transformation learnable equivariant convolutions to dynamically align with each sample's inherent symmetry. Extensive experiments on super resolution, denoising, and deraining validate our theoretical findings and show significant superiority over standard baselines and traditional equivariant models. Our code and supplementary material are available at https://github.com/tanfy929/SA-Conv.

Hyperspectral and multispectral image fusion serves as an effective approach to obtain high-resolution hyperspectral images at low cost. Current deep learning-based hyperspectral image fusion methods often focus on fusing images with a fixed number of spectral bands, making them difficult to adapt to remote sensing images with varying spectral bands. To address this limitation, we propose a novel network framework named arbitrary-band multispectral and hyperspectral images fusion network. Specifically, our method proposes an implicit spectral basis generation block to learn continuous spectral basis vectors that are adaptively sampled according to the wavelength of different bands, thereby generating appropriate spectral basis matrices. We also design an arbitrary-band feature extraction block to effectively extract features from hyperspectral images with arbitrary numbers of spectral bands. Furthermore, we develop a spectral information compensation block to enhance spectral information acquisition. Extensive experimental results demonstrate that our approach not only achieves comparable performance with state-of-the-art methods with fixed number of bands, but also exhibits superior adaptability to process data with arbitrary numbers and positions of spectral bands.

Spatial genome organization plays a crucial regulatory role, but its evolutionary development remains unclear. Leveraging Hi-C data from 1,025 species, we trace the evolutionary trajectories of genome organization through 2 higher-order architectures, “global folding” (spatial organization of the karyotype) and “checkerboard” (spatial organization of chromatin compartments). Earlier unicellular life forms mostly displayed random genome configurations. Throughout the evolution of plants, global folding became and remained the prominent architecture. However, animals progressively developed more pronounced checkerboard architectures; these are also apparent during early embryogenesis, which suggests that they act as a conserved mechanism of gene regulation. In contrast, plants exhibit comparatively weaker checkerboard patterns and instead preferentially organize co-regulated genes into linear genomic clusters. Both strategies of gene arrangement reinforce the biological principle that “structure determines function”: divergent evolutionary paths converge on architectural solutions that reflect gene regulatory requirements over time.

Image-to-image translation (I2I) is a fundamental task in computer vision, focused on mapping an input image from a source domain to a corresponding image in a target domain while preserving domain-invariant features and adapting domain-specific attributes. Despite the remarkable success of deep learning-based I2I approaches, the lack of paired data and unsupervised learning framework still hinder their effectiveness. In this work, we address the challenge by incorporating transformation symmetry priors into image-to-image translation networks. Specifically, we introduce rotation group equivariant convolutions to achieve rotation equivariant I2I framework, a novel contribution, to the best of our knowledge, along this research direction. This design ensures the preservation of rotation symmetry, one of the most intrinsic and domain-invariant properties of natural and scientific images, throughout the network. Furthermore, we conduct a systematic study on image symmetry priors on real dataset and propose a novel transformation learnable equivariant convolutions (TL-Conv) that adaptively learns transformation groups, enhancing symmetry preservation across diverse datasets. We also provide a theoretical analysis of the equivariance error of TL-Conv, proving that it maintains exact equivariance in continuous domains and provide a bound for the error in discrete cases. Through extensive experiments across a range of I2I tasks, we validate the effectiveness and superior performance of our approach, highlighting the potential of equivariant networks in enhancing generation quality and its broad applicability. Our code is available at https://github.com/tanfy929/Equivariant-I2I

Existing color image denoising methods often fail to adequately capture correlations among RGB channels, leading to structural blurring and the loss of fine details. To overcome this limitation, we propose QMSANet, a Quaternion Multi-Scale Attention Network designed to explicitly model inter-channel correlations (i.e., correlations among RGB channels) throughout the denoising process, thereby enabling stronger noise suppression and more faithful detail reconstruction. Our network is built around three complementary modules: the Quaternion Multi-Scale Sparse Block (QMSB), the Quaternion Stacked Enhancement Block (QSEB), and the Lightweight Quaternion Attention Block (LQAB). These modules form a cohesive processing pipeline. Specifically, the QMSB first extracts sparse multi-scale features, allowing the model to capture contextual information at different granularities. These features are then refined by the QSEB, which enhances deep inter-channel interactions and stabilizes feature propagation to improve representational quality. Finally, the LQAB adapts the refined features through a lightweight attention strategy that selectively highlights the most informative responses with minimal computational overhead. Together, these modules operate sequentially to address key denoising challenges, improving efficiency while reducing incomplete noise removal, detail loss, and edge artifacts. Extensive experiments on standard color image denoising benchmarks show that QMSANet consistently outperforms state-of-the-art models under both synthetic and real-world noise. Moreover, although blind denoisers typically underperform their non-blind counterparts, our blind variant (i.e., QMSANet-B) still surpasses most representative methods.

Multi-dimensional data (e.g., videos and spatial transcriptomics) often exhibit multi-directional local smoothness, a property that is challenging to model accurately. While classical multivariate total variation (TV) methods aim to capture such multi-directional regularity, they tend to introduce discretization errors due to discrete differences or interpolation. In this work, we propose a concise yet effective multivariate neural directional TV framework (termed N-NeurDTV). Specifically, we utilize a tensor function representation parameterized by deep neural networks (DNNs) to continuously represent an N-dimensional signal. The N-NeurDTV leverages multivariate directional derivatives of the DNN outputs w.r.t. input coordinates to capture multi-directional smoothness. By leveraging the subspace orthogonal decomposition of RN, the framework naturally generalizes to multivariate higher-order and space-variant N-NeurDTV, where the dominant direction of N-NeurDTV is adaptively refined during inference. The multivariate N-NeurDTV eliminates the need for discretization and alleviates discretization errors. We demonstrate the effectiveness of N-NeurDTV through theoretical analyses and extensive experimental validations across various data recovery tasks, including multi-dimensional video and magnetic resonance image inpainting, multispectral image denoising, and multi-slice spatial transcriptomics reconstruction, with N-NeurDTV consistently outperforming state-of-the-art methods.

Exploring and modeling the rain generation mechanism is critical for augmenting paired data to ease the training of rainy image processing models. Most of the conventional methods handle this task in an artificial physical rendering manner, through elaborately designing fundamental elements constituting rains. These kinds of methods, however, are over-dependent on human subjectivity, which limits their adaptability to real rains. In contrast, recent deep learning (DL) methods have achieved great success by training a neural network-based generator from pre-collected rainy image data. However, current methods usually design the generator in a “closed box” manner, increasing the learning difficulty and data requirements. To address these issues, this study proposes a novel DL-based rain generator, which fully takes the physical generation mechanism underlying rains into consideration and well encodes the learning of the fundamental rain factors (i.e., shape, orientation, length, width, and sparsity) explicitly into the deep network. Its significance lies in that the generator not only elaborately designs essential elements of the rain to simulate expected rains, like conventional artificial strategies, but also finely adapts to complicated and diverse practical rainy images, like DL methods. By rationally adopting the filter parameterization technique, the proposed rain generator is finely controllable with respect to rain factors and able to learn the distribution of these factors purely from data without the need for rain factor labels. Our unpaired generation experiments demonstrate that the rain generated by the proposed rain generator is not only of higher quality but also more effective for deraining and downstream tasks compared to current state-of-the-art rain generation methods. Besides, the paired data augmentation experiments, including both in-distribution and out-of-distribution (OOD), further validate the diversity of samples generated by our model for in-distribution deraining and OOD generalization tasks.

Self-supervised image denoising methods have garnered significant research attention in recent years, for this kind of method reduces the requirement of large training datasets. Compared to supervised methods, self-supervised methods rely more on the prior embedded in deep networks themselves. As a result, most of the self-supervised methods are designed with Convolution Neural Networks (CNNs) architectures, which well capture one of the most important image prior, translation equivariant prior. Inspired by the great success achieved by the introduction of translational equivariance, in this paper, we explore the way to further incorporate another important image prior. Specifically, we first apply high-accuracy rotation equivariant convolution to self-supervised image denoising. Through rigorous theoretical analysis, we have proved that simply replacing all the convolution layers with rotation equivariant convolution layers would modify the network into its rotation equivariant version. To the best of our knowledge, this is the first time that rotation equivariant image prior is introduced to self-supervised image denoising at the network architecture level with a comprehensive theoretical analysis of equivariance errors, which offers a new perspective to the field of self-supervised image denoising. Moreover, to further improve the performance, we design a new mask mechanism to fusion the output of rotation equivariant network and vanilla CNN-based network, and construct an adaptive rotation equivariant framework. Through extensive experiments on three typical methods, we have demonstrated the effectiveness of the proposed method. The code is available at: https://github.com/liuhanze623/AdaReNet.

Diabetic retinopathy (DR) is a leading cause of vision loss in working-age adults, and analyzing retinal fundus plays a crucial role in early DR screening. However, multi-lesion segmentation of fundus images remains a challenging task due to the highly diverse shape, size, position, and brightness of lesions. In this paper, we propose an interpretable network framework for multi-lesion segmentation by integrating the prior knowledge of retinal fundus images into it. Specifically, we propose a probabilistic model for retinal fundus images, which is in a foreground-background decomposition manner, and both the characteristics of the foreground (lesions) and background (non-lesion fundus images) are taken into consideration. Then, we exploit the Expectation- Maximization (EM) algorithm to solve the proposed model and design a novel network architecture under the guidance of the calculation flow of the algorithm, called Decomposition-Segmentation Network (DS-Net). The components of the network consist of two subnetworks: S-Net and D-Net, corresponding to the E step and M step of the EM algorithm, respectively. D-Net aims to decompose the background and foreground compositions of the fundus image and S-Net takes the foreground as input and executes segmentation task. Similar to the EM algorithm, the two subnetworks separate the original segmentation task into two much easier but interpretable sub-tasks. This not only helps enhance the performance, but also greatly facilitates a deeper analysis of the network. Moreover, the proposed DS-Net framework can be easily integrated with current lesion segmentation networks in a plug-and-play manner, by setting current segmentation networks as S-Net in the framework, and lead to general performance improvements. Experiments on benchmark datasets substantiate the superiority of DS-Net quantitatively and visually. The code of our method is available at https://github.com/tanfy929/DS-Net.

Equivariant and invariant deep learning models have been developed to exploit intrinsic symmetries in data, demonstrating significant effectiveness in certain scenarios. However, these methods often suffer from limited representation accuracy and rely on strict symmetry assumptions that may not hold in practice. These limitations pose a significant drawback for image restoration tasks, which demands high accuracy and precise symmetry representation. To address these challenges, we propose a rotation-equivariant regularization strategy that adaptively enforces the appropriate symmetry constraints on the data while preserving the network's representational accuracy. Specifically, we introduce EQ-Reg, a regularizer designed to enhance rotation equivariance, which innovatively extends the insights of data-augmentation-based and equivariant-based methodologies. This is achieved through self-supervised learning and the spatial rotation and cyclic channel shift of feature maps deduce in the equivariant framework. Our approach firstly enables a non-strictly equivariant network suitable for image restoration, providing a simple and adaptive mechanism for adjusting equivariance based on task. Extensive experiments across three low-level tasks demonstrate the superior accuracy and generalization capability of our method, outperforming state-of-the-art approaches.

Retinal vessel segmentation is of great clinical significance for the diagnosis of many eye-related diseases, but it is still a formidable challenge due to the intricate vascular morphology. With the skillful characterization of the translation symmetry existing in retinal vessels, convolutional neural networks (CNNs) have achieved great success in retinal vessel segmentation. However, the rotation-and-scale symmetry, as a more widespread image prior in retinal vessels, fails to be characterized by CNNs. Therefore, we propose a rotation-and-scale equivariant Fourier parameterized convolution (RSF-Conv) specifically for retinal vessel segmentation and provide the corresponding equivariance analysis. As a general module, RSF-Conv can be integrated into existing networks in a plug-and-play manner while significantly reducing the number of parameters. For instance, we replace the traditional convolution filters in U-Net, Iter-Net, DE-DCGCN-EE, and FR-UNet, with RSF-Convs, and faithfully conduct comprehensive experiments. RSF-Conv-enhanced methods not only have slight advantages under in-domain evaluation but also, more importantly, outperform all comparison methods by a significant margin under out-of-domain evaluation. It indicates that the remarkable generalization of RSF-Conv holds greater practical clinical significance for the prevalent cross-device and cross-hospital challenges in clinical practice. To comprehensively demonstrate the effectiveness of RSF-Conv, we also apply RSF-Conv + U-Net and RSF-Conv + Iter-Net to retinal artery/vein classification and achieve promising performance as well, indicating its clinical application potential. The code is available at https://github.com/szhc0gk/RSF-Conv

Global dependency modeling and spatial position modeling are two core issues of the foundational architecture design in current deep learning frameworks. Recently, Vision Transformers (ViTs) have achieved remarkable success in computer vision, leveraging the powerful global dependency modeling capability of the self-attention mechanism. Furthermore, Mamba2 has demonstrated its significant potential in natural language processing tasks by explicitly modeling the spatial adjacency prior through the structured mask. In this paper, we propose Polyline Path Masked Attention (PPMA) that integrates the self-attention mechanism of ViTs with an enhanced structured mask of Mamba2, harnessing the complementary strengths of both architectures. Specifically, we first ameliorate the traditional structured mask of Mamba2 by introducing a 2D polyline path scanning strategy and derive its corresponding structured mask, polyline path mask, which better preserves the adjacency relationships among image tokens. Notably, we conduct a thorough theoretical analysis on the structural characteristics of the proposed polyline path mask and design an efficient algorithm for the computation of the polyline path mask. Next, we embed the polyline path mask into the self-attention mechanism of ViTs, enabling explicit modeling of spatial adjacency prior. Extensive experiments on standard benchmarks, including image classification, object detection, and segmentation, demonstrate that our model outperforms previous state-of-the-art approaches based on both state-space models and Transformers. For example, our proposed PPMA-T/S/B models achieve 48.7%/51.1%/52.3% mIoU on the ADE20K semantic segmentation task, surpassing RMT-T/S/B by 0.7%/1.3%/0.3%, respectively. Code is available at https://github.com/zhongchenzhao/PPMA.

Against the magnetic resonance imaging (MRI) reconstruction task, current deep learning based methods have achieved promising performance. Nevertheless, most of them are confronted with two main problems: (1) For most current MRI reconstruction methods, the down-sampling pattern is generally preset in advance, which makes it hard to flexibly handle the complicated real scenarios where the training data and the testing data are obtained under different sampling settings, thus constraining the model generalization capability. (2) They have not fully incorporated the physical imaging mechanism between the down-sampling pattern estimation and high-resolution MRI reconstruction into deep network design for this specific task. To alleviate these issues, we propose a model-driven MRI reconstruction network called MXNet, which considers the relationship between the undersampling pattern and imaging by encoding the mask into the network. Specifically, based on the MR physical imaging process, we first jointly optimize the down-sampling pattern and MRI reconstruction network. Then, based on the proposed optimization algorithm and the deep unfolding technique, we correspondingly construct the deep network where the physical imaging mechanism for MRI reconstruction is fully embedded into the entire learning process. Based on different settings between training data and testing data, with both consistent and inconsistent down-sampling patterns, extensive experiments comprehensively substantiate the effectiveness of our proposed MXNet in detail reconstruction as well as its fine generality. Moreover, we provide detailed model analysis and validate that our proposed framework shows fine generality and it can still accomplish superior performance when the downsampling mask is accurately available. The code is available at https://github.com/sunliyangna0705/MXNet.

Online tensor decompositions are powerful and proven techniques that address the challenges in processing high-velocity streaming tensor data, such as traffic flow and weather system. The main aim of this work is to propose a novel online functional tensor decomposition (OFTD) framework, which represents a spatial-temporal continuous function using the CP tensor decomposition parameterized by coordinate-based implicit neural representations (INRs). The INRs allow for natural characterization of continually expanded streaming data by simply adding new coordinates into the network. Particularly, our method transforms the classical online tensor decomposition algorithm into a more dynamic continual learning paradigm of updating the INR weights to fit the new data without forgetting the previous tensor knowledge. To this end, we introduce a long-tail memory replay method that adapts to the local continuity property of INR. Extensive experiments for streaming tensor completion using traffic, weather, user-item, and video data verify the effectiveness of the OFTD approach for streaming data analysis. This endeavor serves as a pivotal inspiration for future research to connect classical online tensor tools with continual learning paradigms to better explore knowledge underlying streaming tensor data.

Fourier ptychography (FP) is a new computational imaging technique with the advantage of being able to provide super-resolution imaging. FP has a very complex degradation process. Merging with Fourier transforms and pupil aperture scanning causes difficulty in reconstructing high-resolution images by the commonly used deep neural network methods, e.g., based on convolutional neural networks (CNNs). In this paper, we propose a new optimization algorithm for FP, which is carefully designed so that it only constrains concise operations. Then, we unfold the proposed algorithm to design a new neural network, MDFP-Net, specifically for the FP task. MDFP-Net is consistent with a few stages, which well corresponds to the iterations of the proposed optimization algorithm for FP. This not only makes MDFP-Net more intuitively interpretable, but also makes MDFP-Net much more suitable for FP tasks than commonly used CNNs. Moreover, we have built a long-distance reflection FP measurement system and tested our neural network in real experiments. Simulation and real experimental results show that the proposed network can provide better reconstruction results than either traditional algorithms or other deep learning methods. Code is available at https://github.com/BP113/MDFPNET.