Reranking is a critical component in many information retrieval pipelines. Despite remarkable progress in text-only settings, multimodal reranking remains challenging, particularly when the candidate set contains hybrid text and image items. A key difficulty is the modality gap: a text reranker is intrinsically closer to text candidates than to image candidates, leading to biased and suboptimal cross-modal ranking. Vision-language models (VLMs) mitigate this gap through strong cross-modal alignment and have recently been adopted to build multimodal rerankers. However, most VLM-based rerankers encode all candidates as images, and treating text as images introduces substantial computational overhead. Meanwhile, existing open-source multimodal rerankers are typically trained on general-domain data and often underperform in domain-specific scenarios. To address these limitations, we propose UniRank, a VLM-based reranking framework that natively scores and orders hybrid text-image candidates without any modality conversion. Building on this hybrid scoring interface, UniRank provides an end-to-end domain adaptation pipeline that includes: (1) an instruction-tuning stage that learns calibrated cross-modal relevance scoring by mapping label-token likelihoods to a unified scalar score; and (2) a hard-negative-driven preference alignment stage that constructs in-domain pairwise preferences and performs query-level policy optimization through reinforcement learning from human feedback (RLHF). Extensive experiments on scientific literature retrieval and design patent search demonstrate that UniRank consistently outperforms state-of-the-art baselines, improving Recall@1 by 8.9
Cross-modal matching tasks have achieved significant progress, yet remain limited by mini-batch subsampling and scarce labelled data. Existing objectives, such as contrastive losses, focus solely on instance-level alignment and implicitly assume that all feature dimensions contribute equally. Under small batches, this assumption amplifies noise, making alignment signals unstable and biased. We propose DART (Dual-level Alignment via Robust Transport), a framework that augments instance-level alignment with feature-level regularization based on the Unbalanced Wasserstein Distance (UWD). DART constructs reliability-weighted marginals that adaptively reweight channels according to their cross-modal consistency and variance statistics, highlighting stable and informative dimensions while down-weighting noisy or modality-specific ones. From a theoretical perspective, we establish concentration bounds showing that instance-level objectives scale with the maximum distance across presumed aligned pairs, while feature-level objectives are governed by the Frobenius norm of the transport plan. By suppressing unmatched mass and sparsifying the transport plan, DART reduces the effective transport diameter and tightens the bound, yielding greater robustness under small batches. Empirically, DART achieves state-of-the-art retrieval performance on three audio-text benchmarks, with particularly strong gains under scarce labels and small batch sizes.
Self-limited epilepsy with centrotemporal spikes (SeLECTS) is the most common form of focal epilepsy in childhood, accounting for 20-25 % of all childhood epilepsy cases and may be associated with cognitive dysfunction and behavioral issues. Accurate detection and assessment of epileptic discharges in EEG signals, particularly the spike-wave index (SWI), are crucial for timely intervention and treatment. Manual analysis of EEG data is laborintensive and prone to errors, underscoring the need for automated methods. In the present study, we propose a novel Dual-Stream Spatial-Spectral-Temporal Large model (DeaSTL) that leverages a large-scale EEG architecture to effectively capture the multidimensional characteristics of EEG signals associated with SeLECTS syndrome. Our model integrates multi-view temporal representations and spatial-spectral representations through a dual-stream approach, enhancing the learning of complex patterns in EEG data. We introduce the SJTU SeLECTS EEG Dataset (SLED), a comprehensive EEG dataset from 212 patients diagnosed with SeLECTS, including annotations for abnormal discharge detection, wake-sleep period classification, and SWI estimation. Addressing the previously unexplored problem of SWI prediction, we provide a novel method for quantifying the severity of epileptic discharges during sleep. Extensive experiments demonstrate that our DeaSTL model significantly outperforms several state-of-the-art methods across multiple tasks, showcasing its potential for clinical application in assisting diagnosis and treatment planning.
Glioblastoma (GBM) remains the most aggressive tumor, urgently requiring novel therapeutic strategies. Here, we present a dry-to-wet framework combining generative modeling and experimental validation to optimize peptides targeting ATP5A, a potential peptide-binding protein for GBM. Our framework introduces the first lead-conditioned generative model, which focuses exploration on geometrically relevant regions around lead peptides and mitigates the combinatorial complexity of de novo methods. Specifically, we propose POTFlow, a Prior and Optimal Transport-based Flow-matching model for peptide optimization. POTFlow employs secondary structure information (e.g. helix, sheet, and loop) as geometric constraints, which are further refined by optimal transport to produce shorter flow paths. With this design, our method achieves state-of-the-art performance compared with five popular approaches. When applied to GBM, our method generates peptides that selectively inhibit cell viability and significantly prolong survival in a patient-derived xenograft model. As the first lead peptide-conditioned flow matching model, POTFlow holds strong potential as a generalizable framework for therapeutic peptide design.
PIWI proteins maintain genome integrity by piRNA-guided cleavage of complementary RNA targets. While Cleave-N'-Seq (CNS-seq) has advanced our understanding of PIWI targeting logic through quantitative mapping of cleavage rates and pairing rules, its labor-intensive workflows hinder systematic exploration of sequence determinants. Here, we present PAIRNet, a deep learning framework that predicts PIWI-mediated RNA cleavage rates by explicitly modeling guide-target interactions. Recognizing that interaction geometry, not just sequence, dictates cleavage efficiency, PAIRNet integrates biochemical insights with computational innovation: it encodes pairing states, mismatch types, insertions, and deletions alongside learnable positional embeddings to quantify spatial dependencies; employs a hybrid CNN-Transformer architecture prioritizing duplex dynamics over static sequence features to resolve both local catalytic motifs (e.g., contiguous base-pairing at g10-g11) and distal structural perturbations; and incorporates interpretability modules (saliency maps, counterfactual analysis) to link interaction patterns to biochemical insights and uncover position-specific cleavage rules. Validated across four PIWI-guide datasets, PAIRNet consistently ranks among the top two performers in all experimental conditions, achieving the most pronounced relative improvements in PCC, 34.7% for MILI and 14.6% for MIWI, over second-ranking methods. Critically, PAIRNet recapitulates key biological principles-stringent complementarity at catalytic residues (g10-g11) and tolerance for 3' mismatches-aligning with structural studies of PIWI dynamics. By bridging biochemical precision with computational scalability, PAIRNet establishes a roadmap for designing high-specificity piRNA silencing tools while accelerating mechanistic studies of RNA-guided genome defense.
Electroencephalography (EEG) plays a vital role in clinical and cognitive applications such as epilepsy diagnosis and emotion recognition. However, the low signal-to-noise ratio, inter-subject variability, and inherent non-stationarity of EEG signals present substantial modeling challenges. While recent Transformer-based models offer promising long-range modeling capabilities, their self-attention mechanism behaves as a low-pass filter, suppressing high-frequency neural patterns critical for decoding transient events. In this work, we provide the first formal analysis demonstrating this low-pass behavior in self-attention mechanisms when applied to EEG signals, revealing a fundamental limitation of deep attention-based EEG models. To address this, we propose SEBSFormer, a spectral-enhanced bi-Stream Transformer that jointly models temporal dependencies and spectral structures. SEBSFormer integrates three key modules: a spectral compensation module that restores high-frequency components via residual correction in the Fourier domain; a multi-scale temporal attention module for saliency-guided temporal compression; and a graph-guided dynamic fusion module for adaptive spatial aggregation across electrodes. Extensive experiments on three benchmark datasets—TUAB, TUEV, and SEED—demonstrate that SEBSFormer consistently outperforms existing state-of-the-art models across both clinical and affective tasks. Our findings establish a new paradigm for frequency-aware EEG modeling.
Intracranial aneurysms are life-threatening and require accurate, timely detection. Traditional manual diagnosis by radiologists can be subjective, leading to misdiagnoses, while existing deep learning approaches struggle with small aneurysms or cases complicated by surrounding tissues. In this paper, we present the Multi-View Refinement Network (MVRNet), a framework that delivers clinically meaningful performance gains on the most challenging intracranial aneurysm cases. It directly addresses real-world diagnostic difficulties, particularly for small, obscured, or ambiguously bounded lesions that conventional methods often miss. In particular, we propose a feature enhancement technique to obtain boundary-clear CTA, an informative supplement to the bone-free CTA, to further reduce the impact of noise and enhance the robustness of IA recognition. Moreover, we develop a multi-view encoder with 2D slicing in different directions to mitigate tissue occlusion effects, coupled with a progressively refined decoder that iteratively corrects uncertain predictions, ensuring precise localization and segmentation. Experimental results indicate that MVRNet improves the F1-score by 24% and the IoU by 16% in comparison with the state-of-the-art methods. Remarkably, previous methods struggled with challenging cases, while our method still achieves satisfactory results, demonstrating a >2× improvement over prior arts on challenging test sets. Our code has been released in https://github.com/TheResearchWorks/MVRNet.
Graph Alignment (GA) is an NP-hard combinatorial challenge. The existing methods usually work on small-scale graphs because their computational complexity grows as the square of the number of nodes. Recent techniques relied on alignment labels to divide the large source and target graphs into small ones respectively, which are then aligned separately. However, the alignment labels in expansive real-world graphs are often scarce, which makes GA even more challenging. To address these issues, we propose a novel self-supervised learning framework that is able to work on the graphs of million nodes without any alignment labels, where the existing methods usually fail to output an answer within a reasonable time. Our method also falls in the divide-and-conquer paradigm. We do not cluster the source and target graph separately by the existing graph clustering algorithms, because it is difficult to pair and then align the cluster-induced source and target subgraphs in the absence of alignment labels. We devise an Alignment-Aware Clustering (AAC) method to partition the source and target nodes jointly and keep the ought-to-be-aligned pairs of nodes within the same cluster as much as possible. Furthermore, we develop a Topology-Aware Repartition (TAR) that leverages pseudo-alignment labels as anchors for re-clustering, preserving detailed structural information within clusters. This enables graph neural networks to effectively enhance the graph representation learning for node alignments which are self-supervised by a graph-matching solver. Extensive experiments have demonstrated that our method greatly reduces computational demands and sustains high alignment accuracies in large-scale graph applications.
Predicting the synergy of drug combinations is crucial for cancer treatment and drug development. Accurate prediction requires the integration of multiple types of data, including molecular structures of individual drugs, available synergy scores between drugs, and gene expression information from different cancer cell lines. The first two types contain multi-scale information within or between drugs, while the cell lines serve as the contextual background for drug interactions. Existing machine learning methods fail to fully utilize and integrate these information, leading to suboptimal performance. To address this issue, we introduce GraphFusion, an innovative approach that combines molecular graphs and drug synergy graphs with cell line contextual information. By employing novel GCN and Graphormer modules capable of accepting and utilizing external information, GraphFusion integrates these two levels of graph information. Specifically, the molecular graphs pass fine-grained structural information to the synergy graphs, while the synergy graphs convey global drug interaction data to the molecular graphs. Additionally, cell line information is incorporated as contextual background. This comprehensive integration enables GraphFusion to achieve state-of-the-art results on the O'Neil and NCI-ALMANAC datasets.
General intelligence requires quick adaption across tasks. While existing reinforcement learning (RL) methods have made progress in generalization, they typically assume only distribution changes between source and target domains. In this paper, we explore a wider range of scenarios where not only the distribution but also the environment spaces may change. For example, in the CoinRun environment, we train agents from easy levels and generalize them to difficulty levels where there could be new enemies that have never occurred before. To address this challenging setting, we introduce a causality-guided self-adaptive representation-based approach, called CSR, that equips the agent to generalize effectively across tasks with evolving dynamics. Specifically, we employ causal representation learning to characterize the latent causal variables within the RL system. Such compact causal representations uncover the structural relationships among variables, enabling the agent to autonomously determine whether changes in the environment stem from distribution shifts or variations in space, and to precisely locate these changes. We then devise a three-step strategy to fine-tune the causal model under different scenarios accordingly. Empirical experiments show that CSR efficiently adapts to the target domains with only a few samples and outperforms state-of-the-art baselines on a wide range of scenarios, including our simulated environments, CartPole, CoinRun and Atari games.
PIWI-interacting RNAs (piRNAs) are critical genome guardians that guide PIWI proteins to cleave trans-posable element transcripts. Predicting these cleavage events is vital for understanding gene regulation but is challenged by the complex, “relaxed” rules of piRNA targeting and the noisy cellular context of in vivo data. Previous models, trained solely on in vivo data, struggle with high noise, non-functional binding events, and binary outputs that fail to capture the quantitative biophysics of piRNA-target interactions. Recognizing that precise, quantitative in vitro data directly quantifies biophysical parameters of cleavage, we aimed to leverage this information to overcome the noise and complexity inherent in in vivo observations. To achieve this, we developed piR-DANN, a deep learning framework based on a novel, biologically-informed adversarial domain adaptation strategy. Our core innovation moves beyond conventional approaches by providing the domain classifier with structural and sequence determinants of targeting summarizing known biological rules, in addition to the deep features learned by the model. This asymmetric design compels the feature extractor to learn the fundamental, domain-invariant principles of piRNA targeting. piR-DANN outperforms existing benchmarks predictive performance, with AUROC scores of 94.7% and 99.7% on two independent test sets. Furthermore, counterfactual analysis reveals a positional importance map concordant with PIWI catalytic core constraints, validating its extraction of biological signal from noise. By integrating heterogeneous data into an accurate, interpretable framework, our work explores piRNA biology and proposes a generalizable approach for deciphering gene regulatory systems.
In recent years, neural network-guided heuristic search algorithms, such as Monte-Carlo tree search and A$^\*$ search, have achieved significant advancements across diverse practical applications. Due to the challenges stemming from high state-space complexity, sparse training datasets, and incomplete environmental modeling, heuristic estimations manifest uncontrolled inherent biases towards the actual expected evaluations, thereby compromising the decision-making quality of search algorithms. Sampling exploration enhanced A$^\*$ (SeeA$^\*$) was proposed to improve the efficiency of A$^\*$ search by constructing an dynamic candidate subset through random sampling, from which the expanded node was selected. However, uniform sampling strategy utilized by SeeA$^\*$ facilitates exploration exclusively through the injection of randomness, which completely neglects the heuristic knowledge relevant to open nodes. Moreover, the theoretical support of cluster sampling remains ambiguous. Despite the existence of potential biases, heuristic estimations still encapsulate certain valuable information. In this paper, epistemic exploratory A$^\*$ search (KeeA$^\*$) is proposed to integrate heuristic knowledge for calibrating the sampling process. We first theoretically demonstrate that SeeA$^\*$ with cluster sampling outperforms uniform sampling due to the distribution-aware selection with higher variance. Building on this insight, cluster scouting and path-aware sampling are introduced in KeeA$^\*$ to further exploit heuristic knowledge to increase the sampling mean and variance, respectively, thereby generating higher-quality extreme candidates and enhancing overall decision-making performance. Finally, empirical results on retrosynthetic planning and logic synthesis demonstrate superior performance of KeeA$^*$ compared to state-of-the-art heuristic search algorithms.
Accurately predicting drug synergy in cancer therapy remains challenging due to the strong dependence of drug effectiveness on cell-line context. Most existing models overlook this variability and fail to fully capture drug-cell line interactions. We present CATSyn, a Context-Aware heTerogeneous graph model for synergistic drug combination prediction. CATSyn introduces a context-aware attention mechanism that dynamically adjusts network weights based on cell-line environments, capturing cellspecific drug effects while maintaining generalization. To model these effects, we construct heterogeneous graphs that integrate drug and cell-line features into composite nodes, supported by universal nodes to share global information. Experiments on benchmark datasets demonstrate that CATSyn achieves state-of-the-art performance in both standard and unseen cell-line settings, highlighting its ability to balance specificity and generalization in synergy prediction.
The effectiveness of model training heavily relies on the quality of available training resources. However, budget constraints often impose limitations on data collection efforts. To tackle this challenge, we introduce causal exploration in this paper, a strategy that leverages the underlying causal knowledge for both data collection and model training. We, in particular, focus on enhancing the sample efficiency and reliability of the world model learning within the domain of task-agnostic reinforcement learning. During the exploration phase, the agent actively selects actions expected to yield causal insights most beneficial for world model training. Concurrently, the causal knowledge is acquired and incrementally refined with the ongoing collection of data. We demonstrate that causal exploration aids in learning accurate world models using fewer data and provide theoretical guarantees for its convergence. Empirical experiments, on both synthetic data and real-world applications, further validate the benefits of causal exploration. The source code is available at https://github.com/CMACH508/CausalExploration.
Free-hand sketch, as a versatile medium of communication, can be viewed as a collection of strokes arranged in a spatial layout to convey a concept. Due to the abstract nature of the sketches, changes in stroke position may make them difficult to recognize. Recently, Graphic sketch representations are effective in representing sketches. However, existing methods overlook the significance of the spatial layout of strokes and the phenomenon of strokes being drawn in the wrong positions is common. Therefore, we developed a self-supervised task to correct stroke placement and investigate the impact of spatial layout on learning sketch representations. For this task, we propose a spatially aware method, named SketchGloc, utilizing multiple graphs for graphic sketch representations. This method utilizes grids for each stroke to describe the spatial layout with other strokes, allowing for the construction of multiple graphs. Unlike other methods that rely on a single graph, this design conveys more detailed spatial layout information and alleviates the impact of misplaced strokes. The experimental results demonstrate that our model outperforms existing methods in both our proposed task and the traditional controllable sketch synthesis task. Additionally, we found that SketchGloc can learn more robust representations under our proposed task setting. The source code is available at https://github.com/CMACH508/SketchGloc.
Monte-Carlo tree search (MCTS) and reinforcement learning contributed crucially to the success of AlphaGo and AlphaZero, and A$^*$ is a tree search algorithm among the most well-known ones in the classical AI literature. MCTS and A$^*$ both perform heuristic search and are mutually beneficial. Efforts have been made to the renaissance of A$^*$ from three possible aspects, two of which have been confirmed by studies in recent years, while the third is about the OPEN list that consists of open nodes of A$^*$ search, but still lacks deep investigation. This paper aims at the third, i.e., developing the Sampling-exploration enhanced A$^*$ (SeeA$^*$) search by constructing a dynamic subset of OPEN through a selective sampling process, such that the node with the best heuristic value in this subset instead of in the OPEN is expanded. Nodes with the best heuristic values in OPEN are most probably picked into this subset, but sometimes may not be included, which enables SeeA$^*$ to explore other promising branches. Three sampling techniques are presented for comparative investigations. Moreover, under the assumption about the distribution of prediction errors, we have theoretically shown the superior efficiency of SeeA$^*$ over A$^*$ search, particularly when the accuracy of the guiding heuristic function is insufficient. Experimental results on retrosynthetic planning in organic chemistry, logic synthesis in integrated circuit design, and the classical Sokoban game empirically demonstrate the efficiency of SeeA$^*$, in comparison with the state-of-the-art heuristic search algorithms.
Recent sketch synthesis methods have demonstrated the capability of generating lifelike outcomes. However, these methods directly encode the entire sketches making it challenging to decouple the strokes from the sketches and have difficulty in controlling local sketch synthesis, e.g., stroke editing. Besides, the sketch editing task encounters the issue of accurately positioning the edited strokes, because users may not be able to draw on the exact position and the same stroke may appear in various locations in different sketches. We propose SketchEdit to realize flexible editing of sketches at the stroke-level for the first time. To tackle the challenge of decoupling strokes, SketchEdit divides a drawing sequence of a sketch into a series of strokes based on the pen state, aligns the stroke segments to have the same starting position, and learns the embeddings of every stroke by a proposed stroke encoder. Moreover, we overcome the problem of stroke placement via a diffusion process, which progressively generates the locations for the strokes to be synthesized, using the stroke features as the guiding condition. Experiments demonstrate that SketchEdit is effective for stroke-level sketch editing and sketch reconstruction. The source code is publicly available at https://github.com/CMACH508/SketchEdit/.
Graph neural networks (GNNs) have emerged as powerful approaches to learn knowledge about graphs and vertices. The rapid employment of GNNs poses requirements for processing efficiency. Due to incompatibility of general platforms, dedicated hardware devices and platforms are developed to efficiently accelerate training and inference of GNNs. We conduct a survey on hardware acceleration for GNNs. We first include and introduce recent advances of the domain, and then provide a methodology of categorization to classify existing works into three categories. Next, we discuss optimization techniques adopted at different levels. And finally we propose suggestions on future directions to facilitate further works.
Objective. In helical tomotherapy, image-guided radiotherapy employs megavoltage computed tomography (MVCT) for precise targeting. However, the high voltage of megavoltage radiation introduces substantial noise, significantly compromising MVCT image clarity. This study aims to enhance MVCT image quality using a deep learning-based denoising method. Approach. We propose an unpaired MVCT denoising network using a coupled generative adversarial network framework (DeCoGAN). Our approach assumes that a universal latent code within a shared latent space can reconstruct any given pair of images. By employing an encoder, we enforce this shared-latent space constraint, facilitating the conversion of low-quality (noisy) MVCT images into high-quality (denoised) counterparts. The network learns the joint distribution of images from both domains by leveraging samples from their respective marginal distributions, enhanced by adversarial training for effective denoising. Main Results. Compared to an analytical algorithm (BM3D) and three deep learning-based methods (RED-CNN, WGAN-VGG and CycleGAN), the proposed method excels in preserving image details and enhancing human visual perception by removing most noise and retaining structural features. Quantitative analysis demonstrates that our method achieves the highest peak signal-to-noise ratio and Structural Similarity Index Measurement values, indicating superior denoising performance. Significance. The proposed DeCoGAN method shows remarkable MVCT denoising performance, making it a promising tool in the field of radiation therapy.
Finding known homologs of newly predicted proteins is essential for understanding their functions and mechanisms. It is a highly complex task because proteins undergo various changes during evolution. Traditional methods based on sequence or structure alignment either have low accuracy or take a long time. Recent deep learning-based methods primarily focus on structural information, yet they can’t fully exploiting protein information. To solve this problem, in this paper, we propose a novel general-purpose protein embedding model that can be used for homology search. It first employs a protein language pre-trained model to extract protein sequence embeddings, capturing intricate biological patterns. Subsequently, a Transformer integrating protein structural information generates the high-level representations. By combining protein sequence and structural features, the model can effectively exploit the rich contextual and spatial information inherent in proteins. We applied the model to the SCOP dataset for protein superfamily classification, achieving a classification accuracy of 86.97%, outperforming state-of-the-art method by 7.91%. The source code has been published on GitHub (https://github.com/CMACH508/PEST).