The detection head constitutes a pivotal component within object detectors, tasked with executing both classification and localization functions. Regrettably, the commonly used parallel head often lacks omni perceptual capabilities, such as deformation perception (DP), global perception (GP), and cross-task perception (CTP). Despite numerous methods attempting to enhance these abilities from a single aspect, achieving a comprehensive and unified solution remains a significant challenge. In response to this challenge, we develop an innovative detection head, termed UniHead, to unify three perceptual abilities simultaneously. More precisely, our approach: 1) introduces DP, enabling the model to adaptively sample object features; 2) proposes a dual-axial aggregation transformer (DAT) to adeptly model long-range dependencies, thereby achieving GP; and 3) devises a cross-task interaction transformer (CIT) that facilitates interaction between the classification and localization branches, thus aligning the two tasks. As a plug-and-play method, the proposed UniHead can be conveniently integrated with existing detectors. Extensive experiments on the COCO dataset demonstrate that our UniHead can bring significant improvements to many detectors. For instance, the UniHead can obtain +2.7 AP gains in RetinaNet, +2.9 AP gains in FreeAnchor, and +2.1 AP gains in GFL. The code is available at https://github.com/zht8506/UniHead.

Underwater acoustic target recognition (UATR) can be significantly empowered by advancements in deep learning (DL). However, the effectiveness of DL-based UATR methods is often constrained by the limited computing resources available on underwater platforms. Most of the existing knowledge distillation (KD) strategies try to build lightweight DL models, but these strategies rarely consider the acoustic properties of underwater environments, making them less efficient for UATR tasks. Thus, fully harnessing the potential of DL techniques while ensuring the model's practicality, is one of the urgent problems to be solved in UATR research. In this work, we introduce the union-domain KD (UDKD) to establish an accurate and lightweight UATR model. UDKD integrates two KD strategies: dual-frequency band distillation (DBD) and cross-domain masked distillation (CMD). DBD improves the learning process for a simple student model by decoupling the knowledge of spectrograms into the local structural (i.e., line spectra) and global composition (i.e., propagation patterns) aspects. CMD reduces redundant information from the Fourier Transform process, enabling the student model to concentrate on essential signal elements and to learn underlying time-frequency distribution. Extensive experiments on two real-world oceanic datasets confirm the superior performance of UDKD compared to existing KD methods, i.e., achieving an accuracy of 94.81% (up arrow 3.19% versus 91.62%). Notably, UDKD showcases a 10.5% improvement in the prediction accuracy of the lightweight student model.

Underwater images are important in a range of image-driven applications, such as marine biology and underwater surveillance. However, underwater imaging is subject to several factors that can severely degrade image quality, i.e., light absorption and scattering within the water column. An effective underwater image quality assessment (UIQA) metric is therefore needed to accurately quantify image quality, subsequently facilitating the follow-up of underwater vision tasks. In this article, we propose a novel feature-interaction-based UIQA framework, namely, SISC, which addresses the challenges of training data scarcity and complex underwater degradation conditions. A feature refinement module is dedicatedly designed based on self-attention to implement local and nonlocal cross-spatial feature interactions. In addition, we enhance the refined features in a cross-scale fashion using upsampling and downsampling strategies based on cross-attention. With the two stages of feature refinement and feature enhancement, the proposed SISC achieves data-efficient learning and superior performance compared to existing state-of-the-art UIQA and natural IQA (images captured in air) methods, indicating its effectiveness in extracting quality-aware features from underwater images.

Concealed object segmentation (COS) is a challenging task that involves localizing and segmenting those concealed objects that are visually blended with their surrounding environments. Despite achieving remarkable success, existing COS segmenters still struggle to achieve complete segmentation results in extremely concealed scenarios. In this paper, we propose a Hierarchical Coherence Modeling (HCM) segmenter for COS, aiming to address this incomplete segmentation limitation. In specific, HCM promotes feature coherence by leveraging the intra-stage coherence and cross-stage coherence modules, exploring feature correlations at both the single-stage and contextual levels. Additionally, we introduce the reversible re-calibration decoder to detect previously undetected parts in low-confidence regions, resulting in further enhancing segmentation performance. Extensive experiments conducted on three COS tasks, including camouflaged object detection, polyp image segmentation, and transparent object detection, demonstrate the promising results achieved by the proposed HCM segmenter.

The current state-of-the-art No-Reference Image Quality Assessment (NR-IQA) methods typically rely on feature extraction from upstream semantic backbone networks, assuming that all extracted features are relevant. However, we make a key observation that not all features are beneficial, and some may even be harmful, necessitating careful selection. Empirically, we find that many image pairs with small feature spatial distances can have vastly different quality scores, indicating that the extracted features may contain a significant amount of quality-irrelevant noise. To address this issue, we propose a Quality-Aware Feature Matching IQA Metric (QFM-IQM) that employs an adversarial perspective to remove harmful semantic noise features from the upstream task. Specifically, QFM-IQM enhances the semantic noise distinguish capabilities by matching image pairs with similar quality scores but varying semantic features as adversarial semantic noise and adaptively adjusting the upstream task's features by reducing sensitivity to adversarial noise perturbation. Furthermore, we utilize a distillation framework to expand the dataset and improve the model's generalization ability. Our approach achieves superior performance to the state-of-the-art NR-IQA methods on eight standard IQA datasets.

Image Quality Assessment (IQA) models benefit significantly from semantic information, which allows them to treat different types of objects distinctly. Currently, leveraging semantic information to enhance IQA is a crucial research direction. Traditional methods, hindered by a lack of sufficiently annotated data, have employed the CLIP image-text pretraining model as their backbone to gain semantic awareness. However, the generalist nature of these pre-trained Vision-Language (VL) models often renders them suboptimal for IQA-specific tasks. Recent approaches have attempted to address this mismatch using prompt technology, but these solutions have shortcomings. Existing prompt-based VL models overly focus on incremental semantic information from text, neglecting the rich insights available from visual data analysis. This imbalance limits their performance improvements in IQA tasks. This paper introduces an innovative multi-modal prompt-based methodology for IQA. Our approach employs carefully crafted prompts that synergistically mine incremental semantic information from both visual and linguistic data. Specifically, in the visual branch, we introduce a multi-layer prompt structure to enhance the VL model's adaptability. In the text branch, we deploy a dual-prompt scheme that steers the model to recognize and differentiate between scene category and distortion type, thereby refining the model's capacity to assess image quality. Our experimental findings underscore the effectiveness of our method over existing Blind Image Quality Assessment (BIQA) approaches. Notably, it demonstrates competitive performance across various datasets. Our method achieves Spearman Rank Correlation Coefficient (SRCC) values of 0.961(surpassing 0.946 in CSIQ) and 0.941 (exceeding 0.930 in KADID), illustrating its robustness and accuracy in diverse contexts.

Underwater image enhancement (UIE) poses challenges due to distinctive properties of the underwater environment, including low contrast, high turbidity, visual blurriness, and color distortion. In recent years, the application of deep learning has quietly revolutionized various areas of scientific research, including UIE. However, existing deep learning-based UIE methods generally suffer from issues of weak robustness and limited adaptability. In this paper, inspired by residual and attention mechanisms, we propose a more reliable and reasonable UIE network called RAUNE-Net by employing residual learning of high-level features at the network's bottle-neck and two aspects of attention manipulations in the down-sampling procedure. Furthermore, we collect and create two datasets specifically designed for evaluating UIE methods, which contains different types of underwater distortions and degradations. The experimental validation demonstrates that our method obtains promising objective performance and consistent visual results across various real-world underwater images compared to other eight UIE methods. Our example code and datasets are publicly available at https://github.com/fansuregrin/RAUNE-Net.

Blind Image Quality Assessment (BIQA) aims to simulate human assessment of image quality. It has a great demand for labeled data, which is often insufficient in practice. Some researchers employ unsupervised methods to address this issue, which is challenging to emulate the human subjective system. To this end, we introduce a unified framework that combines semi-supervised and incremental learning to address the mentioned issue. Specifically, when training data is limited, semi-supervised learning is necessary to infer extensive unlabeled data. To facilitate semi-supervised learning, we use knowledge distillation to assign pseudo-labels to unlabeled data, preserving analytical capability. To gradually improve the quality of pseudo labels, we introduce incremental learning. However, incremental learning can lead to catastrophic forgetting. We employ Experience Replay by selecting representative samples during multiple rounds of semi-supervised learning, to alleviate forgetting and ensure model stability. Experimental results show that the proposed approach achieves state-of-the-art performance across various benchmark datasets. After being trained on the LIVE dataset, our method can be directly transferred to the CSIQ dataset. Compared with other methods, it significantly outperforms unsupervised methods on the CSIQ dataset with a marginal performance drop (-0.002) on the LIVE dataset. In conclusion, our proposed method demonstrates its potential to tackle the challenges in real-world production processes.

Storing intermediate frame segmentations as memory for long-range context modeling, spatial-temporal memory-based methods have recently showcased impressive results in semi-supervised video object segmentation (SVOS). However, these methods face two key limitations: 1) relying on non-local pixel-level matching to read memory, resulting in noisy retrieved features for segmentation; 2) segmenting each object independently without interaction. These shortcomings make the memory-based methods struggle in similar object and multi-object segmentation. To address these issues, we propose a query modulation method, termed QMVOS. This method summarizes object features into dynamic queries and then treats them as dynamic filters for mask prediction, thereby providing high-level descriptions and object-level perception for the model. Efficient and effective multi-object interactions are realized through inter-query attention. Extensive experiments demonstrate that our method can bring significant improvements to the memory-based SVOS method and achieve competitive performance on standard SVOS benchmarks. The code is available at https://github.com/zht8506/QMVOS.

This paper presents the Frequency-Integrated Visual-Language Network (FIVLNet), a deep learning (DL) framework tailored to improve the diagnostic accuracy for Attention Deficit Hyperactivity Disorder (ADHD) using magnetic resonance imaging (MRI) scans. Traditional DL approaches in ADHD diagnosis often overlook the sequential dependencies of MRI images or fail to adequately capture their complex structural details, resulting in suboptimal classification accuracy. To address this, the proposed FIVLNet synergistically integrates both high and low-frequency data from MRI images, based on the Convolutional Neural Network (CNN) and the cross-attention mechanism, subsequently achieving more comprehensive representations of the MRI images. Furthermore, in order to enrich the model's learning capacity, textual embeddings from Contrastive Language-Image Pre-training (CLIP) are introduced to provide additional modalities of information. FIVLNet also preserves a lightweight architecture, which necessitates a smaller number of learnable parameters compared to existing models.

The Gaussian processes (GPs) model the acoustic field in the ocean waveguide by exploiting the correlation of the acoustic field at different receiving depths. Therefore, for the measured acoustic field of a sparsely distributed array, the GPs can predict the dense field at the virtual receivers. The kernel function reflects the correlation of field measurements between different receiving depths. This paper proposes a kernel function based on the modal depth functions of normal modes (NMBK), which is used for Gaussian process regression for denoising and interpolation. The predicted field is then combined with the matched field processing (MFP) method for passive source localization. Replicas are also calculated by an acoustic propagation model at the dense receiving depths. Both simulated data and real data from the SWellEx-96 Event S5 environment are used to verify the validity of the proposed method. Compared with the traditional MFP method, the MFP method combined with the GPs has better localization performance and lower sidelobes on the ambiguity surface. Moreover, the proposed NMBK better describes the characteristics of the ocean waveguide compared to the radial basis function. Therefore, it has better acoustic field prediction performance and makes significant improvements on the MFP method with fewer ambiguous positions.

Underwater image quality assessment (UIQA) is critical to many underwater application scenarios, including marine biology research, marine resource development, underwater exploration, and more. Due to the different attenuation rates of light at different wavelengths and the effects of the absorption and scattering of light by suspended particles in the water, there are many types of distortion in the acquired underwater images. Most underwater images often show color casts, reduced contrast, low brightness, blurred object edges, local texture distortion, etc. degradation phenomena compared to natural images. This renders many of the image quality assessment (IQA) methods designed for natural images inapplicable to underwater images. Currently, there is a lack of UIQA methods that are accurate and efficient. In this paper, we proposed an Attention-Based Underwater Image Quality Evaluator (AUIQE), a novel end-to-end IQA approach suitable for UIQA tasks. Specifically, we introduced channel and spatial dual attention mechanisms on the basis of the distortion characteristics of underwater images to make the network focus on some channels and spatial regions that are relevant to image quality. A large number of experiments were designed and carried out on an underwater image quality assessment dataset, and the experimental results indicate that the prediction performance of AUIQE outperforms some of the latest IQA and UIQA methods. The code of AUIQE will be available at https://github.com/ibaochao/AUIQE .

Blind Image Quality Assessment (BIQA) aims to evaluate image quality in line with human perception, without reference benchmarks. Currently, deep learning BIQA methods typically depend on using features from high-level tasks for transfer learning. However, the inherent differences between BIQA and these high-level tasks inevitably introduce noise into the quality-aware features. In this paper, we take an initial step towards exploring the diffusion model for feature denoising in BIQA, namely Perceptual Feature Diffusion for IQA (PFD-IQA), which aims to remove noise from quality-aware features. Specifically, (i) We propose a {Perceptual Prior Discovery and Aggregation module to establish two auxiliary tasks to discover potential low-level features in images that are used to aggregate perceptual text conditions for the diffusion model. (ii) We propose a Perceptual Prior-based Feature Refinement strategy, which matches noisy features to predefined denoising trajectories and then performs exact feature denoising based on text conditions. Extensive experiments on eight standard BIQA datasets demonstrate the superior performance to the state-of-the-art BIQA methods, i.e., achieving the PLCC values of 0.935 ( vs. 0.905 in KADID) and 0.922 ( vs. 0.894 in LIVEC).

Blind Image Quality Assessment (BIQA) mirrors subjective made by human observers. Generally, humans favor comparing relative qualities over predicting absolute qualities directly. However, current BIQA models focus on mining the "local" context, i.e., the relationship between information among individual images and the absolute quality of the image, ignoring the "global" context of the relative quality contrast among different images in the training data. In this paper, we present the Perceptual Context and Sensitivity BIQA (CSIQA), a novel contrastive learning paradigm that seamlessly integrates "global'' and "local'' perspectives into the BIQA. Specifically, the CSIQA comprises two primary components: 1) A Quality Context Contrastive Learning module, which is equipped with different contrastive learning strategies to effectively capture potential quality correlations in the global context of the dataset. 2) A Quality-aware Mask Attention Module, which employs the random mask to ensure the consistency with visual local sensitivity, thereby improving the model's perception of local distortions. Extensive experiments on eight standard BIQA datasets demonstrate the superior performance to the state-of-the-art BIQA methods.

Language conditioned 3D object grounding aims to find the object within the 3D scene mentioned by natural language descriptions, which mainly depends on the matching between visual and natural language. Considerable improvement in grounding performance is achieved by improving the multimodal fusion mechanism or bridging the gap between detection and matching. However, several mismatches are ignored, i.e., mismatch in local visual representation and global sentence representation, and mismatch in visual space and corresponding label word space. In this paper, we propose crossmodal match for 3D grounding from mitigating these mismatches perspective. Specifically, to match local visual features with the global description sentence, we propose BEV (Bird’s-eye-view) based global information embedding module. It projects multiple object proposal features into the BEV and the relations of different objects are accessed by the visual transformer which can model both positions and features with long-range dependencies. To circumvent the mismatch in feature spaces of different modalities, we propose crossmodal consistency learning. It performs cross-modal consistency constraints to convert the visual feature space into the label word feature space resulting in easier matching. Besides, we introduce label distillation loss and global distillation loss to drive these matches learning in a distillation way. We evaluate our method in mainstream evaluation settings on three datasets, and the results demonstrate the effectiveness of the proposed method.

Blind Image Quality Assessment (BIQA) mirrors subjective made by human observers. The Human Visual System (HVS) assesses image quality by combining a global perspective of the contrasting relationships among samples of varying quality with a local analysis of individual images. However, current BIQA methodologies tend to emphasize local evaluations but overlook the contrasting relationship inherent in global perception, leading to the incomprehensive representation of human subjective assessment. Consequently, the representation learning of the BIQA model remains suboptimal. To address this, we present the Perceptual Context and Sensitivity in BIQA (CSIQA), a novel metric learning paradigm that seamlessly integrates efficient human-centric global and local evaluations into the BIQA methodology. Specifically, the CSIQA comprises two primary components: 1). A Quality Context Contrastive Learning module, that is equipped with different contrastive learning strategies to effectively capture potential quality correlations in the \textbf{global context} of the dataset. 2). A Quality-aware mask attention module, which employs the random masking mechanism to ensure the consistency with visual \textbf{local sensitivity}, thereby improving the model's perception of local distortions. Extensive experiments on eight standard BIQA datasets demonstrate the superior performance to the state-of-the-art BIQA methods, \emph{i.e.,} achieving the PLCC values of 0.941 ($\uparrow 3.3\%$ vs. 0.908 in TID2013) and 0.920 ($\uparrow 2.6\%$ vs. 0.894 in LIVEC).

Underwater image quality assessment (UIQA) plays a crucial role in monitoring and detecting the quality of acquired underwater images in underwater imaging systems. Currently, the investigation of UIQA encounters two major challenges. First, a lack of large-scale UIQA databases for benchmarking UIQA algorithms remains, which greatly restricts the development of UIQA research. The other limitation is that there is a shortage of effective UIQA methods that can faithfully predict underwater image quality. To alleviate these two challenges, in this paper, we first construct a large-scale UIQA database (UIQD). Specifically, UIQD contains a total of 5369 authentic underwater images that span abundant underwater scenes and typical quality degradation conditions. Extensive subjective experiments are executed to annotate the perceived quality of the underwater images in UIQD. Based on an in-depth analysis of underwater image characteristics, we further establish a novel baseline UIQA metric that integrates channel and spatial attention mechanisms and a transformer. Channel- and spatial attention modules are used to capture the image channel and local quality degradations, while the transformer module characterizes the image quality from a global perspective. Multilayer perception is employed to fuse the local and global feature representations and yield the image quality score. Extensive experiments conducted on UIQD demonstrate that the proposed UIQA model achieves superior prediction performance compared with the state-of-the-art UIQA and IQA methods.

Matched field processing (MFP) is a key technique for passive underwater source localization, estimating the source position by matching array measurements with acoustic model replicas. Its effectiveness relies on matching environmental parameters with the actual oceanic environment, but performance declines with environmental mismatches and lower signal-to-noise ratios. This paper proposes a novel approach that integrates neural networks (NNs) and complex Gaussian processes with modal depth functions for acoustic field reconstruction that is more accurate and efficient compared to Gaussian process regression. A meta-learning strategy is used to optimize parameters of the NN. The reconstructed data are denoised and interpolated, generating densely populated acoustic fields at virtual arrays, which are then used as data in MFP. Replicas are also computed at the virtual receivers. This mode-informed complex-valued neural processes enhance MFP performance, particularly in low SNR and mismatched environments. It captures the propagation properties of underwater acoustic fields more comprehensively, showing superior localization performance in both simulated and real-world data from the SWellEx-96 Event S5 environment.

Image Quality Assessment (IQA) and Image Aesthetic Assessment (IAA) aim to simulate human subjective perception of image visual quality and aesthetic appeal. Despite distinct learning objectives, they have underlying interconnectedness due to consistent human assessment perception. In this paper, we propose Unified vision-language pre-training of Quality and Aesthetics (UniQA}), to extract useful and common representations from two tasks, thereby benefiting them simultaneously. However, the lack of text in the IQA datasets and the textual noise in the IAA datasets pose severe challenges for multimodal pre-training. To address this, we (1) utilize multimodal large language models (MLLMs) to generate high-quality text descriptions; (2) use the generated text for IAA as metadata to purify noisy IAA data. To effectively adapt the pre-trained UniQA to downstream tasks, we further propose a lightweight adapter that utilizes versatile cues to fully exploit the extensive knowledge of the pre-trained model. UniQA demonstrates high competitiveness in various image assessment tasks, including classical IQA and IAA tasks, few-label IQA, and other downstream tasks, showing promise as a foundational assessment model. Codes are available at https://github.com/zht8506/UniQA.

The common computational pathology tissue slides mainly include Fresh Frozen (FF) slides and Formalin-Fixed Paraffin-Embedded (FFPE) slides. The FF slide has low quality but is easy to prepare, while the FFPE one is the opposite, which is widely used in pathological slide preservation and high-precision diagnosis. Our goal is to generate FFPE images based on FF patches in order to quickly obtain high-quality slides in a short time and to provide doctors with convenient and accurate diagnostic evidence. We propose ST-MKSC, an FF2FFPE image translation network, which contains a Multi-frequency Domain Hierarchical Constraint (MDHC) network based on key consistent information constraints to keep the content information from the FF domain being preserved in the FFPE domain and Released Constraint loss (RC loss) to weaken the existing constraints, so as to reduce the impact of source domain (FF) style information on the appearance of the target domain (FFPE). We conduct FF2FFPE translation experiments on the TCGA-KIRC Dataset and our method is the best among the existing methods. Our model can effectively eliminate or weaken the cavity, artifacts, and other unreasonable structures in FF slides, and generate high-quality FFPE images.