INTRODUCTION/OBJECTIVE:Traditional drug discovery methods face efficiency bottlenecks in predicting drug-target binding affinity (DTA), particularly for kinase inhibitor screening. This study proposes GTDDTA-a novel deep learning framework based on graph transformers and self-attention mechanisms-to address feature integration deficiencies and stereochemical representation limitations in kinase-targeted DTA prediction. METHODS:Drug molecules were converted into graph structures using RDKit (atoms as nodes, bonds as edges). Proteins were modeled through a dual-path approach: when crystal structures were available, residue contact maps were constructed from heavy-atom coordinates extracted via Biopython with a 5.0 Å cutoff; otherwise, binarized Pconsc4-predicted contact maps were employed. Feature extraction utilized dual graph transformer layers to capture global topological dependencies in drug and target graphs, while a self-attention decoder dynamically weighted critical interaction features. The model underwent rigorous five-fold cross-validation on kinasespecific datasets (Davis and KIBA) using protein-family and molecular-scaffold partitioning strategies, with evaluation metrics including MSE, CI, Pearson correlation, and r²m. RESULTS:GTDDTA achieved breakthrough kinase-specific performance: on the Davis dataset, MSE=0.224 (CI=0.896, Pearson=0.852) and on KIBA, MSE=0.146 (CI=0.897, Pearson=0.887). Generalization validation revealed key findings: cross-protein validation (20% kinase holdout) yielded MSE=0.3863, approaching Landrum's experimental noise threshold, while crossscaffold validation (20% Murcko cluster holdout) showed elevated MSE=0.7455, highlighting chemical space generalization limits. Without data augmentation, the model outperformed mainstream baselines, surpassing ColdDTA by 1.7% and reducing DGraphDTA's error by 24.8%. DISCUSSION:GTDDTA successfully modeled conserved kinase features (e.g., VAIK homology motifs in ATP-binding pockets) through graph transformers, achieving prediction accuracy near experimental variation limits. However, 2D graph descriptors failed to encode stereochemical information (affecting 32% of chiral ligands in Davis), significantly increasing prediction errors for novel scaffolds. This limitation aligns with the fundamental challenge in kinase DTA prediction: balancing global topology modeling with 3D conformational constraints. The study further confirmed that self-attention mechanisms outperform traditional concatenation or crossattention in feature fusion quality. CONCLUSION:This research establishes a new state-of-the-art paradigm for kinase-specific DTA prediction: GTDDTA enables robust generalization across homologous targets through architectural innovations (graph transformers and self-attention fusion), outperforming data augmentation- dependent advanced methods. Future integration of 3D geometric learning will overcome stereochemical representation barriers, extending the model's utility to non-kinase targets.

Aerial Visual Place Recognition (VPR) is critical for Unmanned Aerial Vehicles (UAVs) localization, especially in environments with unstable or unavailable GPS signals. While neural network-based VPR methods have become mainstream, they face significant challenges on UAV platforms. Traditional CNN-based VPR models are highly sensitive to image rotation, degrading their performance in aerial-domain environments. Meanwhile, Transformer-based models have high computational complexity, making them less suitable for resource-constrained UAVs. In this letter, we propose a lightweight, rotation-invariant aerial VPR method. Our approach combines a rotation-equivariant backbone network with a rotation-invariant aggregation layer to ensure descriptor consistency across different orientations. Additionally, we propose an unsupervised training strategy that constructs higher-dimensional descriptors to optimize the model, while maintaining the lower descriptor dimensionality during application. Experimental results show that our method outperforms state-of-the-art methods across multiple aerial VPR datasets. The code will be released at https://github.com/cbbhuxx/UltraVPR.

Rockbursts are a significant geological hazard in deep underground engineering, and accurate short-term risk prediction can mitigate safety risks to personnel and equipment. However, challenges remain in the intelligent processing of microseismic(MS) data and effective rockburst prediction. This study focuses on the Dahongshan Copper Mine, utilizing MS monitoring, the synthetic minority oversampling technique and edited nearest neighbours (SMOTE-ENN), and ensemble learning techniques. We developed an intelligent MS event recognition model and a short-term rockburst risk assessment model, enhancing the automation and efficiency of MS event recognition. Additionally, a short-term rockburst risk assessment system (program) was implemented in the Python programming environment, enabling one-click evaluation and warning of short-term rockburst risks (24 h). The research results indicate that the MS event recognition model achieved an F1 score of 0.9819 and an area under the curve (AUC) value of 0.9989, with improvements of up to 49.63% and 11.24% compared to the original dataset. The rockburst early warning model V-soft achieves peak accuracy and F1 scores of 0.9394 and 0.9173, respectively, demonstrating performance improvements of 6.59-15.68% in accuracy and 14.11-27.21% in F1 score compared to conventional machine learning algorithms and ensemble classifiers. This highlights its superior discriminative capability and robustness in predicting high-intensity rockburst events. Applying the rockburst risk assessment system to the mine enables one-click intelligent rockburst early warning.

The success of autoregressive (AR) language models in text generation has inspired the computer vision community to adopt Large Language Models (LLMs) for image generation. However, considering the essential differences between text and image modalities, the design space of language models for image generation remains underexplored. We observe that image tokens exhibit greater randomness compared to text tokens, which presents challenges when training with token prediction. Nevertheless, AR models demonstrate their potential by effectively learning patterns even from a seemingly suboptimal optimization problem. Our analysis also reveals that while all models successfully grasp the importance of local information in image generation, smaller models struggle to capture the global context. In contrast, larger models showcase improved capabilities in this area, helping to explain the performance gains achieved when scaling up model size. We further elucidate the design space of language models for vision generation, including tokenizer choice, model choice, model scalability, vocabulary design, and sampling strategy through extensive comparative experiments. Our work is the first to analyze the optimization behavior of language models in vision generation, and we believe it can inspire more effective designs when applying LMs to other domains. Finally, our elucidated language model for image generation, termed as ELM, achieves state-of-the-art performance on the ImageNet 256*256 benchmark. The code is available at https://github.com/Pepperlll/LMforImageGeneration.git.

Remote sensing images (RSIs) inherently contain rich spectral information across different frequency bands, yet current semantic segmentation networks predominantly focus on spatial feature extraction while neglecting frequency domain analysis, leading to suboptimal segmentation performance. Spectra in different frequency bands and at different scales exhibit complementarity. Reasonable frequency representation can effectively capture inter-category discriminative patterns that are imperceptible in spatial domain. This paper presents a dual-domain learning framework (FCTNet) that synergistically integrates frequency information with convolutional neural network (CNN) and Transformer architecture, enabling comprehensive feature extraction in both spatial and frequency domains. Our encoder employs a hybrid approach combining fast Fourier convolution with central difference convolution, achieving expanded receptive fields while preserving structural details. The decoder features a concurrent interaction architecture comprising a dynamic multi-scale spatial convolution branch and a dual-scale frequency-adjusted Transformer branch, which establish inter-domain communication through cross-attention mechanisms. Notably, the frequency-adjusted Transformer incorporates adaptive frequency allocation that dynamically prioritizes critical frequency components while suppressing non-essential frequency constituents. This mechanism help improve segmentation accuracy and model robustness. Experiments on two benchmark RSI segmentation datasets demonstrate the superior performance of our method, with ablation studies validating the effectiveness of each proposed component.

Nuclear instance segmentation has played a critical role in pathology image analysis. The main challenges arise from the difficulty in accurately segmenting densely over-lapping instances and the high cost of precise mask-level annotations. Existing fully-supervised nuclear instance segmentation methods, such as boundary-based methods, struggle to capture differences between overlapping instances and thus fail in densely distributed blurry regions. They also face challenges transitioning to point supervision, where annotations are simple and effective. Inspired by natural mudslides, we propose a universal method called Mudslide that uses simple representations to characterize differences between different instances and can easily be extended from fully-supervised to point-supervised. Concretely, we introduce a collapse field and leverage it to construct a force map and initial boundary, enabling a distinctive representation for each instance. Each pixel is assigned a collapse force, with distinct directions between adjacent instances. Starting from the initial boundary, Mudslide executes a pixel-by-pixel collapse along various force directions. Pixels that collapse into the same region are considered as one instance, concurrently accounting for both inter-instance distinctions and intra-instance coherence. Experiments on public datasets show superior performance in both fully-supervised and point-supervised tasks.

Proper interaction between visual and semantic features is crucial to obtain a powerful feature representation for scene text recognition (STR). The existing interaction methods usually treat visual and semantic features as distinct tokens and use transformers to learn contextual information and prior language knowledge, and they achieve promising performance for STR task. However, there still remain several issues needed to be further addressed such as the imbalance number and mis-alignment between visual and semantic features, and the necessarily of stacking transformers to a progressive improvement in accuracy. To this aim, this paper proposes a novel interaction manner namely hierarchical visual-semantic interaction (HVSI) which contains three novel modules including a hierarchical visual-semantic interaction module, fusion module, and visual-semantic alignment module. The hierarchical visual-semantic interaction module employs multiple visual-semantic interaction blocks in various scales to enhance the representation power of visual features and semantic features. To better exploit multi-scale visual and semantic features, the fusion module is introduced to fuse multiple semantic features based on attention mechanisms. Furthermore, our HVSI presents a simple plug-in block named visual-semantic alignment module to alleviate mis-alignment of semantic features by mapping them into a unified semantic space, which helps improve the performance of HVSI. Extensive experiments on multiple benchmarks including English and Chinese text recognition datasets show that our method obtains state-of-the-art or competitive performances.

Deep learning-based in silico alternatives have been demonstrated to be of significant importance in the acceleration of the drug discovery process and enhancement of success rates. Cyclin-dependent kinase 12 (CDK12) is a transcription-related cyclin-dependent kinase that may act as a biomarker and therapeutic target for cancers. However, currently, there is no high selective CDK12 inhibitor in clinical development and the identification of new specific CDK12 inhibitors has become increasingly challenging due to their similarity with CDK13. In this study, we developed a virtual screening workflow that combines deep learning with virtual screening tools and can be applied rapidly to millions of molecules. We designed a Transformer architecture Drug-Target Interaction (DTI) model with dual-branched self-supervised pre-trained molecular graph models and protein sequence models. Our predictive model produced satisfactory predictions for various targets, including CDK12, with several novel hits. We screened a large compound library consisting of 4.5 million drug-like molecules and recommended a list of potential CDK12 inhibitors for further experimental testing. In kinase assay, compared to the positive CDK12 inhibitor THZ531, the compounds CICAMPA-01, 02, 03 displayed more effective inhibition of CDK12, up to three times as much as THZ531. The compounds CICAMPA-03, 05, 04, 07 showed less inhibition of CDK13 compare to THZ531. In vitro, the IC50 of CICAMPA-01, 04, 05, 06, 09 was less than 3 μM in the HER2 positive CDK12 amplification breast cancer cell line BT-474. Overall, this study provides a highly efficient and end-to-end deep learning protocol, in conjunction with molecular docking, for discovering CDK12 inhibitors in cancers. Additionally, we disclose five novel CDK12 inhibitors. These results may accelerate the discovery of novel chemical-class drugs for cancer treatment.

Understanding the selectivity mechanisms of inhibitors toward highly similar proteins is very important in new drug discovery. Developing highly selective targeting of leucine-rich repeat kinase 2 (LRRK2) kinases for the treatment of Parkinson's disease (PD) is challenging because of the similarity of the kinase ATP binding pocket. During the development of LRRK2 inhibitors, off-target effects on other kinases, especially TTK and JAK2 kinases, have been observed. As a result, significant time and resources have been devoted to improving the selectivity for the LRRK2 target. DNL201 is an LRRK2 kinase inhibitor entering phase I clinical studies. The experiments have shown that DNL201 significantly inhibits LRRK2 kinase activity, with >167-fold selectivity over JAK2 and TTK kinases. However, the potential mechanisms of inhibitor preferential binding to LRRK2 kinase are still not well elucidated. In this work, to reveal the underlying general selectivity mechanism, we carried out several computational methods and comprehensive analyses from both the binding thermodynamics and kinetics on two representative LRRK2 inhibitors (DNL201 and GNE7915) to LRRK2. Our results suggest that the structural and kinetic differences between the proteins may play a key role in determining the activity of the selective small-molecule inhibitor. The selectivity mechanisms proposed in this work could be helpful for the rational design of novel selective LRRK2 kinase inhibitors against PD.

Rapid and accurate prediction of drug-target affinity can accelerate and improve the drug discovery process. Recent studies show that deep learning models may have the potential to provide fast and accurate drug-target affinity prediction. However, the existing deep learning models still have their own disadvantages that make it difficult to complete the task satisfactorily. Complex-based models rely heavily on the time-consuming docking process, and complex-free models lacks interpretability. In this study, we introduced a novel knowledge-distillation insights drug-target affinity prediction model with feature fusion inputs to make fast, accurate and explainable predictions. We benchmarked the model on public affinity prediction and virtual screening dataset. The results show that it outperformed previous state-of-the-art models and achieved comparable performance to previous complex-based models. Finally, we study the interpretability of this model through visualization and find it can provide meaningful explanations for pairwise interaction. We believe this model can further improve the drug-target affinity prediction for its higher accuracy and reliable interpretability.

Filter pruning is widely adopted to compress and accelerate the Convolutional Neural Networks (CNNs), but most previous works ignore the relationship between filters and channels in different layers. Processing each layer independently fails to utilize the collaborative relationship across layers. In this paper, we intuitively propose a novel pruning method by explicitly leveraging the Filters Similarity in Consecutive Layers (FSCL). FSCL compresses models by pruning filters whose corresponding features are more worthless in the model. The extensive experiments demonstrate the effectiveness of FSCL, and it yields remarkable improvement over state-of-the-art on accuracy, FLOPs and parameter reduction on several benchmark models and datasets.

Parkinson's disease (PD) is the second most common neurodegenerative disorder that affects more than ten million people worldwide. However, the current PD treatments are still limited and alternative treatment strategies are urgently required. Leucine-rich repeat kinase 2 (LRRK2) has been recognized as a promising target for PD treatment. However, there are no approved LRRK2 inhibitors on the market. To rapidly identify potential drug repurposing candidates that inhibit LRRK2 kinase, we report a structure-based drug repurposing workflow that combines molecular docking, recursive partitioning model, molecular dynamics (MD) simulation, and molecular mechanics-generalized Born surface area (MM-GBSA) calculation. Thirteen compounds screened from our drug repurposing workflow were further evaluated through the experiment. The experimental results showed six drugs (Abivertinib, Aumolertinib, Encorafenib, Bosutinib, Rilzabrutinib, and Mobocertinib) with IC50 less than 5 μM that were identified as potential LRRK2 kinase inhibitors. The most potent compound Abivertinib showed potent inhibitions with IC50 toward G2019S mutation and wild-type LRRK2 of 410.3 nM and 177.0 nM, respectively. Our combination screening strategy had a 53% hit rate in this repurposing task. MD simulations and MM-GBSA free energy analysis further revealed the atomic binding mechanism between the identified drugs and G2019S LRRK2. In summary, the results showed that our drug repurposing workflow could be used to identify potent compounds for LRRK2. The potent inhibitors discovered in our work can be a starting point to develop more effective LRRK2 inhibitors.

Semantic information plays an important role in scene text recognition (STR) as well as visual information. Although state-of-the-art models have achieved great improvement in STR, they usually rely on extra external language models to refine the semantic features through context information, and the separate utilization of semantic and visual information leads to biased results, which limits the performance of those models. In this paper, we propose a novel model called Visual-Semantic Transformer (VST) for text recognition. VST consists of several key modules, including a ConvNet, a visual module, two visual-semantic modules, a visual-semantic feature interaction module and a semantic module. VST is a conceptually much simpler model. Different from existing STR models, VST can efficiently extract semantic features without using external language models and it also allows visual features and semantic features to interact with each other parallel so that global information from two domains can be fully exploited and more powerful representations can be learned. The working mechanism of VST is highly similar to our cognitive system, where the visual information is first captured by our sensory organ, and is simultaneously transformed to semantic information by our brain. Extensive experiments on seven public benchmarks including regular/ irregular text recognition datasets verify the effectiveness of VST, it outperformed other 14 popular models on four out of seven benchmark datasets and yielded competitive performance on the other three datasets. © 2022. The copyright of this document resides with its authors. It may be distributed unchanged freely in print or electronic forms. † : Equal contribution; *: Corresponding author. 2 DIAO ET AL. : VISUAL-SEMANTIC TRANSFORMER FOR SCENE TEXT RECOGNITION

Molecular machine learning based on graph neural network has a broad prospect in molecular property identification in drug discovery. Molecules contain many types of substructures that may affect their properties. However, conventional methods based on graph neural networks only consider the interaction information between nodes, which may lead to the oversmoothing problem in the multi-hop operations. These methods may not efficiently express the interacting information between molecular substructures. Hence, We develop a Molecular SubStructure Graph ATtention (MSSGAT) network to capture the interacting substructural information, which constructs a composite molecular representation with multi-substructural feature extraction and processes such features effectively with a nested convolution plus readout scheme. We evaluate the performance of our model on 13 benchmark data sets, in which 9 data sets are from the ChEMBL data base and 4 are the SIDER, BBBP, BACE, and HIV data sets. Extensive experimental results show that MSSGAT achieves the best results on most of the data sets compared with other state-of-the-art methods.(c) 2022 Elsevier Ltd. All rights reserved.

Leucine-rich repeat kinase 2 (LRRK2) has been reported in the pathogenesis of Parkinson's disease (PD). G2019S mutant is the most common pathogenic mutation in LRRK2-related PD patients. Inhibition of LRRK2 kinase activity is proposed to be a new therapeutic approach for PD treatment. Therefore, understanding the molecular basis of the interaction between LRRK2 and its inhibitors will be valuable for the discovery and design of LRRK2 inhibitors. However, the structure of human LRRK2 in complex with the inhibitor has not been determined, and the inhibitory mechanism underlying LRRK2 still needs to be further investigated. In this study, molecular dynamics (MD) simulation combined with the molecular mechanics generalized born surface area (MM-GBSA) binding free energy calculation and pharmacophore modeling methods was employed to explore the critical residues in LRRK2 for binding of inhibitors and to investigate the general structural features of the inhibitors with diverse scaffolds. The results from MD simulations suggest that the hinge region residues Glu1948 and Ala1950 play a significant role in maintaining the intermolecular hydrogen bond interaction with the G2019S LRRK2 protein and inhibitor. The strong hinge hydrogen bond with an occupancy rate of more than 95% represents the high activity of LRRK2 inhibitors, and the hydrogen bond interaction with the kinase catalytic loop region could compromise selectivity. Further pharmacophore modeling reveals that the high activity LRRK2 inhibitor should have one aromatic ring, one hydrogen bond acceptor, and one hydrogen bond donor. Hence, the obtained results can provide valuable information to understand the interactions of LRRK2 inhibitors at the atomic level that will be helpful in designing potent inhibitors of LRRK2.

Anti-cancer drug discoveries have been serendipitous, we sought to present the Open Molecular Graph Learning Benchmark, named CandidateDrug4Cancer, a challenging and realistic benchmark dataset to facilitate scalable, robust, and reproducible graph machine learning research for anti-cancer drug discovery. CandidateDrug4Cancer dataset encompasses multiple most-mentioned 29 targets for cancer, covering 54869 cancer-related drug molecules which are ranged from pre-clinical, clinical and FDA-approved. Besides building the datasets, we also perform benchmark experiments with effective Drug Target Interaction (DTI) prediction baselines using descriptors and expressive graph neural networks. Experimental results suggest that CandidateDrug4Cancer presents significant challenges for learning molecular graphs and targets in practical application, indicating opportunities for future researches on developing candidate drugs for treating cancers.

Large scale pre-training models have been widely used in named entity recognition (NER) tasks. However, model ensemble through parameter averaging or voting can not give full play to the differentiation advantages of different models, especially in the open domain. This paper describes our NER system in the SemEval 2022 task11: MultiCoNER. We proposed an effective system to adaptively ensemble pre-trained language models by a Transformer layer. By assigning different weights to each model for different inputs, we adopted the Transformer layer to integrate the advantages of diverse models effectively. Experimental results show that our method achieves superior performances in Farsi and Dutch.

Contrastive learning has emerged as a powerful tool for graph representation learning. However, most contrastive learning methods learn features of graphs with fixed coarse-grained scale, which might underestimate either local or global information. To capture more hierarchical and richer representation, we propose a novel Hierarchical Contrastive Learning (HCL) framework that explicitly learns graph representation in a hierarchical manner. Specifically, HCL includes two key components: a novel adaptive Learning to Pool (L2Pool) method to construct more reasonable multi-scale graph topology for more comprehensive contrastive objective, a novel multi-channel pseudo-siamese network to further enable more expressive learning of mutual information within each scale. Comprehensive experimental results show HCL achieves competitive performance on 12 datasets involving node classification, node clustering and graph classification. In addition, the visualization of learned representation reveals that HCL successfully captures meaningful characteristics of graphs.