Although existing diffusion-based methods produce visually rich textures at extremely low bitrates, they often sacrifice structural fidelity, resulting in significant deviations from the original image. To address this fundamental trade-off, we propose Fidelity-Perception Diffusion-based Image Compression (FPD-IC), a two-stage conditional diffusion framework that explicitly separates structure reconstruction and detail restoration. In Stage I, we use a VAE-based compressor to recover structurally faithful conditional images from highly compact bitstreams. In Stage II, a diffusion model, guided by the output from Stage I, generates visually rich details. This conditional approach allows the diffusion model to focus exclusively on perceptual enhancement while preserving the overall structure established in Stage I. Additionally, we introduce a lightweight Fidelity-Perception Tuner Module (FPTM) to combine the outputs of both stages, enabling controllable trade-offs between fidelity and perceptual quality. Extensive experiments on the Kodak and Tecnick datasets demonstrate the effectiveness and robustness of FPD-IC. On the Tecnick dataset, FPD-IC outperforms state-of-the-art diffusion-based methods by 2.24-3.57 dB in PSNR at bitrates below 0.06 bpp, while also achieving superior perceptual quality. Furthermore, FPD-IC shows strong robustness to input noise, consistently maintaining high fidelity and perceptual quality under Gaussian perturbations. The code will be released at https://github.com/mlkk518/FPD-IC.
Image-based quantified haze visibility evaluation is crucial for both atmospheric science and computer vision. Existing methods typically rely on datasets with precisely annotated visibility labels. However, the scarcity of real-world labeled data significantly hinders their scalability and practical deployment. To address this, we introduce HZSC, a real-world haze visibility dataset collected from traffic surveillance cameras. It consists of 36 diverse outdoor scenes, each containing 20 images ranked by haze visibility levels. Meanwhile, we propose the haze visibility evaluator (HVE) for referenceless image haze visibility assessment. HVE comprises three key components: a depth feature extractor, a haze visibility feature extractor, and a feature fusion module. The fused features are fed into learned fog-free and foggy Multivariate Gaussian (MVG) models to generate haze visibility scores. For training, We design a scoring-ranking framework using pairwise supervision, adopting a weighted hinge ranking loss to improve discrimination of subtle visibility variations. Extensive experiments on both the public MRFID dataset and our collected HZSC dataset demonstrate that our HVE consistently outperforms existing state-of-the-art methods. Examples of our dataset can be available at https://github.com/ZanChen1/HLVRS.
Existing compressive sensing (CS) reconstruction algorithms are primarily designed for deterministic measurements. However, in real-world scenarios, quantization bit erasures during transmission or storage introduce uncertainty into measurements and significantly complicate reconstruction. We extend the CS framework to explicitly handle such bit-level uncertainty, enabling robust image recovery from quantized measurements with missing bits. We begin by enumerating all valid combinations of the erased quantization bits to construct a candidate set of feasible measurement values. This candidate set is then incorporated as a constraint in a newly formulated inverse problem. We propose an iterative plug-and-play algorithm to solve this problem, alternating between two key steps: (1) an image update using a pretrained denoiser, and (2) a measurement update via a soft-min projection strategy accelerated by coordinate descent. Extensive experiments demonstrate the effectiveness of the proposed approach, achieving high-quality reconstructions even under extremely low sampling rates and severe erasure conditions. Our framework offers a scalable and principled solution for bit-erasure-robust CS reconstruction in error-prone and resource-constrained imaging environments.
Existing image compression algorithms have achieved significant advancements in preserving high fidelity. However, attaining superior perceptual quality often requires retraining a compression model specifically optimized for this objective, which incurs additional training costs and hampers rapid deployment. To address this limitation, we propose CodeIC, an innovative framework that leverages codebook priors and adversarial learning to enhance perceptual quality without the need for retraining existing high-fidelity compression models. CodeIC operates through three meticulously designed stages. First, we pretrain a high-quality discrete codebook using a redesigned vector quantized generative adversarial network (VQGAN), which serves as a robust generative prior. Second, we introduce a Transformer-based prediction model to align the decoded image features from an existing compression algorithm with the frozen high-quality codebook. Finally, we design a hierarchical prior integration network (HPIN), which integrates Swin Transformer blocks (STBs) and multi-head cross-attention modules (MCMs) to query hierarchical priors from the codebook, enhanced by adversarial learning. This approach significantly improves the decoding of texture-rich images, ensuring both perceptual quality and fidelity. Extensive experiments demonstrate that CodeIC achieves a favorable trade-off between perception and fidelity, delivering superior perceptual quality than high-fidelity compressors and higher fidelity than perception-oriented models. More importantly, it substantially boosts downstream task performance (e.g., salient object detection) over baseline methods, proving its preservation of critical semantic information.
Recently, single image compressive sensing has advanced considerably and achieved favorable performance. However, most methods only focus on designing effective reconstruction networks yet ignoring crude sampling strategy, where less effective information in measurement values makes it impossible to reconstruct high-quality images, especially at low sampling rates. To break aforementioned obstacle, we propose a novel pyramid sampling-based compressive sensing network, termed PCSNet, for single image compressive sensing reconstruction. Specifically, the novelties of our proposed PCSNet are mainly two-fold. Firstly, we design a novel pyramid compressive sampling (PCS) scheme to obtain measurement values from image pyramids, which ensures that measurements carry more context and multi-scale information, conducive to precise CS reconstruction. Moreover, a scale assignment module (SAM) is proposed to dynamically stress on the significance of different scales. Secondly, we propose a pyramid reconstruction network (PRN) to be suitable for PCS, which parallelly reconstruct compressive images from multi-scale measurements. An information aggregation module (IAM) is proposed in PRN to mutually supply structural and detail information of different scales. Extensive experimental results on three datasets manifest our proposed PCSNet outperforms other state-of-the-art CS models. The source code can be downloaded via https://github.com/WHK-Huake/PCSNet.
With the development of deep learning in recent years, the performance of object detection under conventional cameras has been significantly improved. Nevertheless, due to the distortion caused by the fisheye cameras, detecting objects in this scenario remains a significant challenge. The dominant approaches focus on modifying the shape of the bounding box to better align the boundaries of the distorted object. However, these methods neglect the learning of spatial distortion information, which prevents them from satisfactory results. In this paper, we propose a novel fisheye camera detection network to learn distortion features better, dubbed SDANet. SDANet is composed of a series of SDABlocks, which are designed to learn spatial distortion features. Each SDABlock consists of multiple convolution kernels of different sizes, and it can generate the most suitable kernel based on the current input's distortion characteristics. Moreover, to address the limitations of the scarcity and uneven spatial distribution of fisheye image datasets on performance improvement, we propose a dedicated data augmentation strategy called Prominent Fisheye Distortion Augmentation (PFDAug). PFDAug can further introduce distortions to fisheye images, effectively alleviating these problems. Experimental results on the CEPDOF, MW-R, HABBOF, LOAF, and FishEye8k fisheye image datasets demonstrate that our method achieves state-ofthe-art performance.
Despite the impressive performance of existing image compression algorithms, they struggle to balance perceptual quality and high image fidelity. To address this issue, we propose a novel invertible neural network-based remote sensing image compression (INN-RSIC) method. Our approach captures the compression distortion from an existing image compression algorithm and encodes it as Gaussian-distributed latent variables using an INN, ensuring that the distortion in the decoded image remains independent of the ground truth. By using the inverse mapping of the INN, we input the decoded image with randomly resampled Gaussian variables, generating enhanced images with improved perceptual quality. We incorporate channel expansion, Haar transformation, and invertible blocks into the INN to accurately represent compression distortion. Additionally, a quantization module (QM) is introduced to mitigate format conversion impact, enhancing generalization and perceptual quality. Extensive experiments show that INN-RSIC achieves superior perceptual quality and fidelity compared to existing algorithms. As a lightweight plug-and-play (PnP) method, the proposed INN-based enhancer can be easily integrated into existing high-fidelity compression algorithms, enabling flexible and simultaneous decoding of images with enhanced perceptual quality.
Compressed sensing (CS) provides a robust and simple framework for compressing images in resource-constrained environments. However, CS-based image coding schemes often have poor rate-distortion (R-D) performance, particularly due to the quantization process. Our research indicates that leveraging the image prior enables the estimation of most significant bits (MSBs) from least significant bits (LSBs), which provides a quantization strategy to improve R-D performance without increasing coding complexity. That is discarding MSBs of measurements, and only transmitting LSBs to the decoder side. At the decoder side, we reconstruct images by solving an inverse-quantization set-constrained CS optimization problem. Our approach further employs a tailored designed deep denoiser as the proximal operator to enhance the reconstructed image quality. Extensive experimental results demonstrate that the proposed scheme achieves satisfactory performance, with promising R-D results (PSNR gains over 1.71 dB than JPEG at 0.50 bpp compression ratio), and robust bit error and loss resilience (reconstructed 29.98 dB even with 50% bit loss at 0.50 bpp compression ratio), meanwhile having lower encoding complexity (less than half encoding time of CCSDS-IDC).
Learning-based image compression algorithms typically focus on designing encoding and decoding networks and improving the accuracy of entropy model estimation to enhance the rate-distortion (RD) performance. However, few algorithms leverage the compression distortion prior from existing compression algorithms to improve RD performance. In this article, we propose a latent diffusion model-based remote sensing image compression (LDM-RSIC) method, which aims to enhance the final decoding quality of RS images by utilizing the generated distortion prior from an LDM. Our approach consists of two stages. In Stage I, a self-encoder learns prior from the high-quality input image. In Stage II, the prior is generated through an LDM, conditioned on the decoded image of an existing learning-based image compression algorithm to be used as auxiliary information for generating the texture-rich enhanced images. To better utilize the prior, a channel attention and gate-based dynamic feature attention module (DFAM) is embedded into a Transformer-based multi-scale enhancement network (MEN) for image enhancement. Extensive experimental results demonstrate that the proposed LDM-RSIC outperforms existing state-of-the-art traditional and learning-based image compression algorithms in terms of both subjective perception and objective metrics. Our project is available at https://github.com/mlkk518/LDM-RSIC
Crowd counting is challenged by large appearance variations of individuals in uncontrolled scenes. Many previous approaches elaborated on this problem by learning multi-scale features and concatenating them together for more impressive performance. However, such a naive fusion is intuitional and not optimal enough for a wide range of scale variations. In this paper, we propose a novel feature fusion scheme, called Scalable Omni-scale Distribution Fusion (SODF), which leverages the benefits of different scale distributions from multi-layer feature maps to approximate the real distribution of target scale. Inspired by Gaussian Mixture Model that surmounts multi-scale feature fusion from a probabilistic perspective, our SODF module adaptively integrate multi-layer feature maps without embedding any multi-scale structures. The SODF module is comprised of two major components: an interaction block that perceives the real distribution and an assignment block which assigns the weights to the multi-layer or multi-column feature maps. The newly proposed SODF module is scalable, light-weight, and plug-and-play, and can be flexibly embedded into other counting networks. In addition, we design a counting model (SODF-Net) with SODF module and multi-layer structure. Extensive experiments on four benchmark datasets manifest that the proposed SODF-Net performs favorably against the state-of-the-art counting models. Furthermore, the proposed SODF module can efficiently improve the prediction performance of canonical counting networks, e.g., MCNN, CSRNet, and CAN.
The complex entanglement between darkness and noise hinders the advance of low-light image enhancement. Most existing methods adopted lightening-then-denoising or embedded a special denoising module into enhancement network without specific noise knowledge as supervision to restore low-light images. However, they either fail to remove the amplified noise or blur the detail information. Against above drawbacks, we propose a novel dual prior guidance method for low-light image enhancement that relights darkness and suppresses noise simultaneously. Concretely, the main novelties of our proposed method are three-fold. Firstly, our formulation originates from a statistic observation that darkness can be disentangled into luminance channel, yet noise still exists each channel when low-light images are transformed from RGB space to YCbCr space. It inspires us to design an ingenious method, extracting noise and darkness, termed END, to enhance low-light images. Secondly, we propose a prior extraction network with prior composition module to extract luminance and noise priors from different channels. Thirdly, an image enhancement network deployed with prior guidance module is proposed to progressively lighten the darkness and remove noise. Extensive experiments on multiple benchmarks demonstrate that our proposed method achieves remarkable performance compared to other state-of-the-art low-light image enhancement methods. The source code and trained model can be found in https://github.com/WHK-Huake/END.
Retinex theory-based low-light image enhancement methods have received increasing attention and achieved tremendous advancements. However, there still exist two seldom-explored issues: 1) The above methods only formally simulate the Retinex decomposition, resulting in lacking explicit interpretability. 2) They usually are performed in single-scale space, leading to suboptimal enhancement results. In this paper, we propose an interpretable Multi-scale Retinex Unfolding Network (MRUNet) for low-light image enhancement, which can tackle both of the aforementioned issues simultaneously. Specifically, we formulate low-light image enhancement as a multi-scale Retinex optimization problem and design an iteration minimization solution to solve it. The optimization solution is further unfolded to fabricate MRUNet, which is empowered with clear physical significance and multi-scale prior knowledge in favor of image enhancement. However, it will aggravate model size and efficiency when exploiting multiple proximal mapping networks to extract multi-scale prior from multi-scale inputs. To surmount the issue, we propose a Scale-Aware Proximal mapping Module (SAPM), which efficiently collect multi-scale prior knowledge via the weight sharing strategy. In SAPM, we tailor a scale-aware transformer to model the specific scale-similarity among different scales. Extensive experiments manifest that MRUNet surpasses other Retinex-based low-light image enhancement methods on multiple benchmarks.
Eliminating reflections caused by incident light interacting with reflective medium remains an ill-posed problem in the image restoration area. The primary challenge arises from the overlapping of reflection and transmission components in the captured images, which complicates the task of accurately distinguishing and recovering the clean background. Existing approaches typically address reflection removal solely in the image domain, ignoring the spectral property variations of reflected light, which hinders their ability to effectively discern reflections. In this paper, we start with a new perspective on spectral learning, and propose the Spectral Codebook to reconstruct the optical spectrum of the reflection image. The reflections can be effectively distinguished by perceiving the wavelength differences between different light sources in the spectrum. To leverage the reconstructed spectrum, we design two spectral prior refinement modules to re-distribute pixels in the spatial dimension and adaptively enhance the spectral differences along the wavelength dimension. Furthermore, we present the Spectrum-Aware Transformer to jointly recover the transmitted content in spectral and pixel domains. Experimental results on three different reflection benchmarks demonstrate the superiority and generalization ability of our method compared to state-of-the-art models.
Supervised scene text segmentation has achieved notable progress in recent years. However, its development is largely constrained by the scarcity of high-quality datasets and the high cost of pixel-level annotations. To address this limitation, we explore few-shot learning for text segmentation and propose TSAL, an attribute-aware few-shot framework that leverages a pre-trained CLIP model to learn transferable text attributes for segmentation. Our framework comprises two complementary branches: I) a Visual-Guided Branch that extracts semantic and textural features for foreground text and background regions, respectively, and II) an Adaptive Prompt-Guided Branch that employs learnable prompt templates to capture diverse text attributes with minimal data dependence. To effectively align textual attributes with visual representations, we further introduce an Adaptive Feature Alignment (AFA) module, which aligns learnable attribute tokens with visual features and prompt prototypes, enabling the model to capture both general and distinctive textual characteristics. As a result, TSAL can accurately segment text regions using only a few annotated samples. Extensive experiments demonstrate that our method achieves state-of-the-art performance across several public text segmentation benchmarks under few-shot settings and exhibits strong generalization to text-related tasks.
To achieve large-scale stitching of the hyperspectral remote sensing images obtained by unmanned aerial vehicles (UAVs) equipped with an acousto-optic tunable filter spectrometer, this study proposes a method based on a feature fusion strategy and a seam-finding strategy using hyperspectral image classification. In the feature extraction stage, SuperPoint deep features from images in different spectral segments of the data cube were extracted and fused. The feature depth matcher, LightGlue, was employed for feature matching. During the data cube fusion stage, unsupervised K-means spectral classification was performed separately on the two hyperspectral data cubes. Subsequently, grayscale transformations were applied to the classified images. A dynamic programming method, based on a grayscale loss function, was then used to identify seams in the transformed images. Finally, the identified splicing seam was applied across all bands to produce a unified hyperspectral data cube. The proposed method was applied to hyperspectral data cubes acquired at specific waypoints by UAVs using an acousto-optic tunable filter spectral imager. Experimental results demonstrated that the proposed method outperformed both single-spectral-segment feature extraction methods and stitching methods that rely on seam identification from a single spectral segment. The improvement was evident in both the spatial and spectral dimensions.
Hyperspectral image data is a form of data that combines images and spectra, and there are information differences between images in different bands when performing cube concatenation of hyperspectral data. A stitching seam search strategy based on hyperspectral spectral image classification is proposed to address the insufficient utilization of spectral dimension information in current data cube stitching methods. The main steps in searching for stitching seams are: Iteratively self-organizing data analysis algorithm (ISODATA) is used to classify two hyperspectral data cubes separately. Perform grayscale changes on the classification result images. Use graph cutting method to search for stitching seams on the transformed image. Apply the stitching seam to all bands to obtain the spliced hyperspectral data. The experimental results of applying this method to unmanned aerial hyperspectral data cubes captured by acousto-optic tunable filter (AOTF) spectral imager at waypoints show that our proposed method has certain advantages in both spatial and spectral dimensions compared to using stitching seams obtained from a single spectral segment image to achieve hyperspectral data cube stitching strategy.
Firstly,the current situation of satellite remote sensing,communication,and computing systems was systematically analyzed,and the real-time remote sensing dilemma existing in the traditional satellite remote sensing systems and the satellite remote sensing communication integrated system architecture was deeply analyzed.Secondly,a multi-satellite and satellite-ground coordination inte-grated system architecture for satellite remote sensing,communication,and computation was put forward,and the key components and the working theories modes of the architecture were introduced in detail.Thirdly,the advantages of this architecture were demonstrated and the feasibility of key technologies was systematically clarified.Finally,the simulation and analysis method based on this architec-ture,which could reach minute-level remote sensing tasks with high timeliness and high potential application value compared with the traditional mode,was elaborated.
Deep unfolding network based Block Compressed Sensing (BCS) methods typically remove some signal and retain certain block artifacts simultaneously during iterative deartifacting, which is unfavorable for signal recovery. To enhance reconstruction performance, based on Learned Denoising Iterative Thresholding (LDIT) algorithm. Pseudo Supervised Attention Short-term Memory and Multi-scale Deartifacting (PSASM-MD) based image BCS, is proposed in this paper. Initially, in each iteration, each image block is denoised separately in parallel using residual networks before being concatenated. Subsequently, in conjunction with the Pseudo-Supervised Attention Module (PSAM), Multi-Scale Deartifacting Network (MSD-Net) is used to perform feature extraction on the concatenated images, enabling more efficient removal of block artifacts and improving the reconstruction performance. In this case, PSAM is utilized to extract useful signal components from the residuals containing block artifacts, transfer the short-term memory to the subsequent iteration to minimize the removal of useful signals. Experimental results demonstrate that this approach outperforms existing state-of-the-art BCS methods both in subjective visual perception and objective evaluation metrics.
Deep learning-based methods have garnered significant attention in remote sensing (RS) image compression due to their superior performance. Most of these methods focus on enhancing the coding capability of the compression network and improving entropy model prediction accuracy. However, they typically compress and decompress each image independently, ignoring the significant inter-image similarity prior. In this paper, we propose a codebook-based RS image compression (Code-RSIC) method with a generated discrete codebook, which is deployed at the decoding end of a compression algorithm to provide inter-image similarity prior. Specifically, we first pretrain a high-quality discrete codebook using the competitive generation model VQGAN. We then introduce a Transformer-based prediction model to align the latent features of the decoded images from an existing compression algorithm with the frozen high-quality codebook. Finally, we develop a hierarchical prior integration network (HPIN), which mainly consists of Transformer blocks and multi-head cross-attention modules (MCMs) that can query hierarchical prior from the codebook, thus enhancing the ability of the proposed method to decode texture-rich RS images. Extensive experimental results demonstrate that the proposed Code-RSIC significantly outperforms state-of-the-art traditional and learning-based image compression algorithms in terms of perception quality. The code will be available at \url{https://github.com/mlkk518/Code-RSIC/