Editing specific attributes of a real photograph while preserving the subject’s identity and the most subtle details remains a fundamental challenge in computer vision. Recent generative diffusion and multimodal models have pushed the boundaries of image synthesis quality; however, they struggle to provide the precise, deterministic, single-attribute control required by many practical applications, and their iterative denoising processes entail a considerable computational cost. On the other hand, GAN-based inversion addresses this gap: by projecting a real image into StyleGAN's de-entangled latent space, specific edits can be applied using simple direction vectors without the need for resampling. However, existing GAN inversion methods are far from efficient; for example, optimization-based approaches fit a latent space for each image, which is accurate but too slow for real-time use. Encoder-based methods replace that loop with a single forward pass, but SOTA designs have become increasingly computationally expensive by stacking multi-stage training processes and auxiliary encoders, inflating the number of parameters far beyond what the task requires. Therefore, we introduce MambaStyle, a single-stage encoder-based framework for GAN inversion and editing that uses vision state-space models (VSSMs) to close this efficiency gap. VSSMs process image features with linear rather than quadratic complexity, keeping the model compact and fast without giving up inversion accuracy or editing quality. Experiments on face and car domains show that MambaStyle achieves a better efficiency-quality trade-off than current SOTA methods, with fewer parameters and lower inference time, making it well-suited for real-time applications.
Gravimetry inversion aims to recover 3D subsurface density-contrast models from surface gravity anomalies, which are fundamental to resource exploration. These measurements are typically sparse and irregularly distributed relative to the 3D model’s dimensionality. Classical inversion approaches stabilize the solution through regularization but depend strongly on the assumed prior and careful hyperparameter tuning and tend to degrade under extreme sparsity; moreover, many alternative pipelines rely on conventional interpolation schemes, such as kriging or RBF-based methods, or directly assume samples on a grid can represent the field without explicitly enforcing physical validity. When a pre-interpolation step is performed, the resulting field is rarely constrained to be consistent with the gravity forward operator, which can introduce artifacts in unobserved regions that propagate to the subsequent inversion. To address these limitations, this work introduces a physics-guided closed-loop that (i) replaces the conventional two-stage pipeline with a joint interpolation-inversion optimization, where a continuous gravity field and the 3D density model are co-estimated under the differentiable forward physics and (ii) introduces auxiliary coordinates to enforce physics consistency beyond the unobserved samples, effectively constraining the reconstructed field to remain within the range of physically plausible forward responses and mitigating non-physical solutions in extremely sparse regimes. The proposed method reduces reconstruction error by 38%–67% compared to a neural baseline and by about 95% compared to a classical inversion approach, yielding more coherent reconstructions with fewer artifacts under undersampling.
Ensuring the authenticity and ownership of digital images is increasingly challenging as modern editing tools enable highly realistic forgeries. Existing image protection systems mainly rely on digital watermarking, which is susceptible to sophisticated digital attacks. To address this limitation, we propose a hybrid optical-digital framework that incorporates physical authentication cues during image formation and preserves them through a learned reconstruction process. At the optical level, a phase mask in the camera aperture produces a Null-space Optical Watermark (NOWA) that lies in the Null Space of the imaging operator and therefore remains invisible in the captured image. Then, a Null-Space Network (NSN) performs measurement-consistent reconstruction that delivers high-quality protected images while preserving the NOWA signature. The proposed design enables tamper localization by projecting the image onto the camera's null space and detecting pixel-level inconsistencies. Our design preserves perceptual quality, resists common degradations such as compression, and establishes a structural security asymmetry: without access to the optical or NSN parameters, adversaries cannot forge the NOWA signature. Experiments with simulations and a prototype camera demonstrate competitive performance in terms of image quality preservation, and tamper localization accuracy compared to state-of-the-art digital watermarking and learning-based authentication methods.
Change detection (CD) in remote sensing images is an important problem requiring the analysis of high-dimensional data such as radar, multispectral, and hyperspectral images. Solutions for radar and multispectral data are available in the literature. However, the use of hyperspectral images is still an area of improvement, not only because the existing methods were designed to handle a reduced number of spectral bands but also because they do not fully exploit the rich spectral information available for CD. Additionally to the use of hyperspectral data, the interest in multimodal CD is that different sensors, such as optical and synthetic aperture radar (SAR), can contribute with essential features improving the generation of high-quality change maps. This paper addresses the multimodal CD problem using hyperspectral and optical/SAR images or pairs of hyperspectral images using a multivariate manifold estimation model based on spectral unmixing. The estimated abundance maps associated with hyperspectral images are used as inputs to the CD strategy, instead of the original hyperspectral data enabling significant dimensionality reduction. Several simulations demonstrate the effectiveness of the proposed approach, achieving, for instance, overall accuracy values above 91%, AUC values up to 0.95, and consistently higher recall compared to competing methods. These results highlight the strong performance of the proposed method while preserving its flexibility to handle any combination of radar, multispectral, and hyperspectral images.
Accurate relative phase estimation is fundamental in multi-agent wireless systems and collaborative beamforming. Classical bounds like the Cramér–Rao bound (CRB) can be overly optimistic in low-SNR or biased-estimation regimes. The Ziv–Zakai bound (ZZB) captures threshold effects but requires numerical integration. We develop and evaluate a Young–Westerberg bound (YWB), a Bayesian lower bound obtained by minimizing the biased deterministic CRB over admissible bias functions on a bounded parameter domain. For angular parameters with an i.i.d. uniform prior on [−π, π] and constant Fisher information (independent AWGN), the YWB admits a closed-form, per-eigenmode Bayesian MSE lower bound and is provably tight at both low and high SNR. We compare YWB against the stochastic CRB and the vector ZZB, and validate via Monte Carlo simulations. Results show that YWB provides the tightest prior-aware benchmark for low SNR relative phase estimation.
Solving imaging inverse problems has usually been addressed by designing proper prior models of the underlying signal. However, minimizing the data fidelity term poses significant challenges due to the ill-conditioned sensing matrix caused by physical constraints in the acquisition system. Thus, preconditioning techniques have been adopted in classical optimization theory to address ill-conditioned data-fidelity minimization by transforming the algorithm gradient step to achieve faster convergence and better numerical stability. We extend the preconditioning concept beyond convergence acceleration and use it to improve reconstruction quality. We introduce DIPA: Distilled Preconditioned Algorithms, where a preconditioning operator (PO) is optimized using teacher-guided distillation criteria. Unlike standard model-compression KD, the teacher and student differ by the sensing operators available during reconstruction: the teacher uses a simulated, better-conditioned, and more informative sensing matrix, whereas the student uses the physically feasible sensing matrix. We design different distillation loss functions to transfer different properties of the teacher algorithm to the preconditioned student. The PO can be linear (L-DIPA), allowing interpretability, or non-linear (N-DIPA), parametrized by a neural network, offering better scalability. We validate the proposed PO design across several imaging modalities, including magnetic resonance imaging, compressed sensing, and super-resolution imaging.
Snapshot compressive imaging has emerged as a powerful framework for acquiring high-dimensional data efficiently in a single exposure. Despite significant advances in spectral and temporal imaging, capturing a full-spectral video in a single snapshot remains challenging, as conventional approaches typically rely on multiple acquisitions to separately capture spectral and temporal dimensions. To address this limitation, we propose a single-shot compressed dynamic color-coded spectral video system that employs a windowed temporal encoding approach that improves pixel-intensity uniformity and dynamic range. The proposed compressive coding scheme synchronizes a liquid crystal tunable filter with a coded aperture device to encode and acquire the spectral video within a fixed exposure time. We develop and implement a plug-and-play alternating-direction multiplier method (PnP-ADMM) to efficiently recover the four-dimensional data cubes. Extensive simulations and proof-of-concept experiments demonstrate the effectiveness of the proposed system in capturing and reconstructing compressed spectral video efficiently.
Ultrasound imaging faces a trade-off between image quality and hardware complexity caused by dense transducers. Sparse arrays are one popular solution to mitigate this challenge. This work proposes an end-to-end optimization framework that jointly learns sparse array configuration and image reconstruction. The framework integrates a differentiable Image Formation Model with a HARD Straight Thought Estimator (STE) selection mask, unrolled Iterative Soft-Thresholding Algorithm (ISTA) deconvolution, and a residual Convolutional Neural Network (CNN). The objective combines physical consistency (Point Spread Function (PSF) and convolutional formation model) with structural fidelity (contrast, Side-Lobe-Ratio (SLR), entropy, and row diversity). Simulations using a 3.5 MHz probe show that the learned configuration preserves axial and lateral resolution with half of the active elements. This physics-guided, data-driven approach enables compact, cost-efficient ultrasound probe design without sacrificing image quality, and it is expandable to 3-D volumetric imaging.
Spectral images are highly valuable for various applications, including environmental monitoring and precision agriculture. However, the high cost of specialized sensors limits the wide use of this technology in numerous applications. Current alternatives to acquire high spatial-spectral resolution spectral images, like Single-Pixel Imaging (SPI) enhanced with Deep Optical Coding Design (DOCD), have limitations due to their non-feedback optical designs, leading to limited image quality, with optimal performance achieved only for the specific scenes used during training. This work reformulates the DOCD framework to handle the scene-driven ordering of the Hadamard basis within the SPI architecture for spectral imaging. Taking into account that SPI usually acquires hundreds of snapshots, our approach introduces a scene-driven ordering of the Hadamard matrix for flexible SPI modulation pattern selection based on scene characteristics in an end-to-end optimization. Simulations on spectral datasets and real test-bed acquisitions demonstrate the effectiveness of the proposed method in improving the quality of VIS and NIR spectral images compared to fixed designs.
Hypercomplex signal processing (HSP) offers powerful tools for analyzing and processing multidimensional signals by explicitly exploiting inter-dimensional correlations through Clifford algebra. In recent years, hypercomplex formulations of the phase retrieval (PR) problem, wheren a complex-valued signal is recovered from intensity-only measurements, have attracted growing interest. Hypercomplex phase retrieval (HPR) naturally arises in a range of optical imaging and computational sensing applications, where signals are often modeled using quaternion- or octonion-valued representations. Similar to classical PR, HPR problems may involve measurements obtained via complex, hypercomplex, Fourier, or other structured sensing operators. These formulations open new avenues for the development of advanced HSP-based algorithms and theoretical frameworks. This chapter surveys emerging methodologies and applications of HPR, with particular emphasis on optical imaging systems.
Self-supervised learning for imaging inverse problems is increasingly important in photon-limited settings, where acquiring clean ground truth is impractical and reconstruction must remain stable under dataset and acquisition shifts. This challenge is amplified under Poisson noise, whose signal-dependent statistics interact with sampling operators (e.g., CFA mosaicing). Meanwhile, foundation vision encoders trained at web scale offer distortion-invariant, content-related representations that generalize well across domains, suggesting a promising route to build priors that transfer beyond the training distribution without expensive fine-tuning. This paper proposes an ADMM-inspired unrolled plug-and-play solver for Poisson inverse problems that decouples a closed-form data-consistency update from a parameter-efficient prior. The prior is implemented as a lightweight decoder operating on frozen CLIP RN50 dense multi-scale features, adapting foundation representations with less trainable parameters. For self-supervision, the method integrates GR2R measurement-domain re-corruption with an Equivariant Imaging regularizer via virtual acquisitions. Experiments on Poisson CFA demosaicing and deblurring show competitive quality, improved robustness under shifts, and self-supervised performance approaching supervised training.
Ensuring the authenticity and integrity of hyperspectral images (HSIs) is crucial for remote sensing applications, as unauthorized spectral modifications can compromise data reliability and decision-making processes. Traditional post-acquisition authentication methods leave hyperspectral data vulnerable to manipulation before applying digital protection mechanisms. We present an acquisition-stage optical authentication framework that embeds an optical key directly in the light path of a line-scan hyperspectral imager before digitization. The optical key is designed from scene spectral statistics and jointly optimized for imperceptibility, manipulation detection, and post-attack spectral restoration in a data-driven manner. We initialize and optimize the optical key using distributions parameterized by per-band spectral variance from the training data, which adaptively scales the embedding strength across bands and improves robustness to variance-driven perturbations. To evaluate its resilience, we simulate an optimal attack scenario in which an adversary randomly alters 20%-80% of pixels within a target class. A detection network achieves a maximum accuracy of 97% under light perturbation and maintains above 92% even at 80% manipulation, while maintaining classification performance within 0.6%-1.2% of unsigned data. The restoration network brings spectral signatures to within 0.9 degrees-1.3 degrees SAM. Experiments on simulated and laboratory datasets show robustness gains of up to 4.7% over traditional methods, with preserved downstream performance.
The primary purpose of seismic acquisition geometry design is to illuminate the subsurface to identify potential energy reservoirs. Traditional acquisition design methods include uniform, random, and jittered patterns. Uniform patterns can be impractical due to physical constraints, and irregular patterns can introduce incoherent noise to the data through aliasing. Deep learning (DL) methods have focused on data-driven approaches, regularized by seismic signal quality reconstruction to design undersampled patterns. However, they frequently overlook imaging performance under geological variability. In this work, we introduce imaging-guided seismic pattern optimization (ISPO), a novel DL-based framework. ISPO requires a reference velocity model, from which a forward operator generates modeled shot gathers. A denoising diffusion probabilistic model (DDPM) then produces structurally diverse yet geologically consistent perturbations to the velocity model. These perturbations are used to train an imaging or inversion operator that maps shot gathers to velocity models. The central objective is to optimize receiver and/or source positions using a binary layer, guided by the estimated quality of the velocity models obtained from the optimized pattern, while allowing for undersampling. ISPO was evaluated across three acquisition scenarios-receiver-only, source-only, and simultaneous undersampling. ISPO consistently outperforms both classical and DL-based baselines, including uniform, jittered, random, and end-to-end reconstruction-driven designs. The method yields superior performance in structural similarity index (SSIM), mean-squared error (mse), and perceptual metrics, while preserving key structural features of the velocity model even at low acquired rates.
Gravimetric and magnetic surveys provide complementary information for subsurface characterization. However, their sparse, irregular sampling complicates accurate interpolation. Classical methods, such as kriging and cokriging, rely on predefined covariance models and fixed spatial neighborhoods, often failing to capture non-stationary structures and complex cross-variable interactions. We therefore propose a joint graph-based interpolation framework that learns end-to-end both the spatial graph topology and the interpolation function directly from the data. Survey locations are modeled as nodes in a learned graph with adaptive connectivity, enabling anisotropic and data-driven neighborhoods. Gravimetric and magnetic data are estimated jointly using a shared graph, allowing cross-field information transfer while preserving variable-specific behavior. Graph-based Dirichlet and biharmonic regularization enforce spatial smoothness and structural consistency and robust loss functions enhance stability under noisy observations. Experiments on field geophysical data demonstrate that the proposed method outperforms state-of-the-art interpolation techniques, particularly in data-limited scenarios.
Inverse problems in imaging are ill-posed, leading to infinitely many solutions consistent with the measurements due to the non-trivial null-space of the sensing matrix. Common image priors promote solutions on the general image manifold, such as sparsity, smoothness, or score function. However, as these priors do not constrain the null-space component, they can bias the reconstruction. Thus, we aim to incorporate meaningful null-space information in the reconstruction framework. Inspired by smooth image representation on graphs, we propose Graph-Smooth Null-Space Representation (GSNR), a mechanism that imposes structure only into the invisible component. Particularly, given a graph Laplacian, we construct a null-restricted Laplacian that encodes similarity between neighboring pixels in the null-space signal, and we design a low-dimensional projection matrix from the p-smoothest spectral graph modes (lowest graph frequencies). This approach has strong theoretical and practical implications: i) improved convergence via a null-only graph regularizer, ii) better coverage, how much null-space variance is captured by p modes, and iii) high predictability, how well these modes can be inferred from the measurements. GSNR is incorporated into well-known inverse problem solvers, e.g., PnP, DIP, and diffusion solvers, in four scenarios: image deblurring, compressed sensing, demosaicing, and image super-resolution, providing consistent improvement of up to 4.3 dB over baseline formulations and up to 1 dB compared with end-to-end learned models in terms of PSNR.
Gravimetry images subsurface density contrasts associated with geological structures, geothermal systems, and intrusive bodies. Recovering a three-dimensional density model from gravity observations is highly ill-posed because of its non-uniqueness, limited data coverage, and the attenuation of the gravity field with depth. Classical inversion methods rely on explicit regularization and parameter tuning, whereas supervised deep-learning approaches require representative gravity–density pairs that are rarely available. This paper proposes an unsupervised depth-aware implicit neural representation for 3D gravity inversion. The density volume is represented by multiple coordinate-based neural networks assigned to overlapping depth slabs and optimized directly from the observed gravity measurements through the sensitivity matrix. Slab-specific Fourier features, physics-based depth gains, and scheduled regularization provide structural priors without requiring labeled density models. Experiments on four synthetic scenarios show that the proposed method provides better overall performance in terms of RMSE, PSNR, and SSIM than the evaluated conventional and neural baselines. It also recovers more compact and spatially coherent density bodies, improves the separation of nearby anomalies, preserves internal structures, and reconstructs their vertical extent better. These results indicate that the proposed depth-aware formulation helps to mitigate the depth ambiguity inherent in gravity inversion. In the field experiment, where no ground-truth density model was available, the method produced compact, separated, and vertically coherent anomalies consistent with the observed gravity pattern.
Imaging inverse problems are commonly addressed by minimizing measurement consistency and signal prior terms. While huge attention has been paid to developing high-performance priors, even the most advanced signal prior may lose its effectiveness when paired with an ill-conditioned sensing matrix that hinders convergence and degrades reconstruction quality. In optimization theory, preconditioners allow improving the algorithm's convergence by transforming the gradient update. Traditional linear preconditioning techniques enhance convergence, but their performance remains limited due to their dependence on the structure of the sensing matrix. Learning-based linear preconditioners have been proposed, but they are optimized only for data-fidelity optimization, which may lead to solutions in the null-space of the sensing matrix. This paper employs knowledge distillation to design a nonlinear preconditioning operator. In our method, a teacher algorithm using a better-conditioned (synthetic) sensing matrix guides the student algorithm with an ill-conditioned sensing matrix through gradient matching via a preconditioning neural network. We validate our nonlinear preconditioner for plug-and-play FISTA in single-pixel, magnetic resonance, and super-resolution imaging tasks, showing consistent performance improvements and better empirical convergence.
We present a hyperspectral image (HSI) classifier that fuses a Vision Transformer (ViT) with a semantic index encoder. The encoder computes physically grounded remotesensing indices (e.g., NDVI, NDWI, NDRE, NDBI) using wavelength windows. This yields a compact numeric semantic embedding that is linearly fused with ViT logits. The architecture is end-to-end trainable, numerically stable, requires no textual tokenization, and yields optional humanreadable pixel narratives at inference. Experiments on six HS public datasets demonstrate consistent gains over a ViT baseline, highlighting the benefit of integrating physics-inspired indices into transformer-based HSI classification.
Recent advances in forensic detection techniques, such as the Diffusion Noise Feature (DNF), have enabled highly accurate identification of AI -generated images by exploiting subtle statistical patterns introduced during the denoising process of diffusion models. In this work, we propose an antiforensic diffusion framework that regularizes the image generation process to produce images with DNF representations that closely resemble those of real images. Our approach integrates a DNF-guided regularization term into the reverse diffusion process, effectively minimizing the statistical distance between the generated and real image DNF distributions. Experimental results demonstrate that our method significantly reduces the detection accuracy of state-of-the-art forensic detectors, demonstrating a more realistic image generation.
Phase retrieval (PR) reconstructs phase information from magnitude measurements, known as coded diffraction patterns (CDPs), whose quality depends on the number of snapshots captured using coded phase masks. High-quality phase estimation requires multiple snapshots, which is not desired for efficient PR systems. End-to-end frameworks enable joint optimization of the optical system and the recovery neural network. However, their application is constrained by physical implementation limitations. Additionally, the framework is prone to gradient vanishing issues related to its global optimization process. This paper introduces a Knowledge Distillation (KD) optimization approach to address these limitations. KD transfers knowledge from a larger, lower-constrained network (teacher) to a smaller, more efficient, and implementable network (student). In this method, the teacher, a PR system trained with multiple snapshots, distills its knowledge into a single-snapshot PR system, the student. The loss functions compare the CPMs and the feature space of the recovery network. Simulations demonstrate that this approach improves reconstruction performance compared to a PR system trained without the teacher's guidance.