Accurate prediction of a publicly traded company's fundamentals, especially Earnings Per Share (EPS), is a major step towards evaluating its financial health, and investment potential. EPS prediction is an important task, and similar to other financial prediction tasks, EPS prediction can also benefit from machine and deep learning methods. Here, we come up with 2 deep learning-based methods to forecast a company's EPS from historical attributes including analyst estimates: 1-EPS-ML (EPS Prediction via Multi-frequency LSTM (Long Short Term Memory)), 2-EPS-CL (EPS Prediction via CNN-LSTM (Convolutional Neural Network-Long Short Term Memory)). Our diverse set of attributes ranges from publicly available company fundamentals to other broader financial metrics such as inflation rate. Our multi-frequency LSTM solution is motivated by its ability to efficiently integrate distinct frequencies. On the other hand, CNN-LSTM type architecture is motivated by LSTM's ability to model sequences and CNN's ability to extract patterns in complex data more accurately. We evaluate our approaches over forecasting liquid S&P 500 companies EPS. Across a distinct set of scenarios and conditions, EPS-ML and EPS-CL outperform all competing approaches, including recently-proposed transformer-based Informer and Autoformer, in terms of both machine learning-based and financial evaluation, which underperformance could be due to our shorter input window length (four quarters). Our results are consistent over a distinct set of performance metrics and across various industry sectors. Among different windowing techniques, we find the approach using previous consecutive quarters to perform the best. In summary, we believe our approach could improve decision-making for investors and financial analysts by generating more accurate EPS estimates. To our best knowledge, EPS prediction via time-series deep learning techniques has not been investigated in depth. Our code and datasets are anonymously available at https://github.com/seferlab/epspredict.

Network alignment is a fundamental task in biological network analysis, aiming to identify corresponding nodes across multiple graphs such as protein-protein interaction (PPI) networks. Existing alignment methods struggle to capture the heterophily, hierarchical organization, and higher-order structures commonly observed in real biological networks. To address these challenges, we propose HMGA (Hyperbolic Graph Learning for Multi-Perspective Graph Alignment), a unified framework that integrates hyperbolic and Euclidean representation learning within hypergraph modeling to take into account higher-order structures. It also utilizes a heterophily-aware graph attention network to model heterophily and heterogeneity. HMGA simultaneously encodes first-order, power-law, and higher-order topological features by leveraging hyperbolic geometry to mitigate embedding distortion in scale-free structures while maintaining Euclidean consistency for regular subgraphs. The model fuses multiple alignment matrices generated from distinct embedding spaces to achieve robust cross-network correspondence. To discuss the advantages of the proposed approach, we bring together a fair evaluation framework that systematically compares the performance of different network alignment approaches. Experiments on both synthetic and real-world biological datasets, including Human-Yeast, C. jejuni-E. coli, and A. thaliana-D. melanogaster PPIs, demonstrate that HMGA performs reasonably with respect to state-of-the-art baselines in TOP-K accuracy. Furthermore, HMGA successfully identifies known and novel functional orthologs across distant eukaryotes, validating its biological relevance. Even though many graph alignment methods have been proposed recently, there is no recent systematic comparison of all these methods, which is another contribution of this study.

Cancer is a major threat to human health, and cancer immunotherapy is considered the most promising treatment option. It offers high effectiveness and precision, with fewer side effects than traditional treatments. Tumor T cell antigens (TTCA) are the proteins or protein fragments that stand on the surface of cancer cells and that are recognized by the immune system. However, physical experimental approaches to infer them might be costly and time-consuming, even though they are important for cancer immunotherapy. We propose DeepTHybrid to predict TTCAs by combining chemically hand-designed features with deep protein representations from ProtBERT, an adapted version of the BERT large Language Model, for finding complex representations from the protein sequences. It is the first framework to incorporate these protein embeddings with complementary chemical features for this problem. After the feature extraction step, among multiple classifiers tested, we find Inception-based neural network to outperform the rest, so DeepTHybrid includes it. We evaluate the prediction performance on multiple datasets, where DeepTHybrid consistently outperforms the competing approaches across all datasets. For instance, on the first dataset, DeepTHybrid achieves an accuracy of 0.76, an F1-score of 0.78, outperforming existing machine learning and state-of-the-art TTCA predictors. Across both datasets, combining deep protein embeddings with chemical attributes yields superior performance compared to using either feature type alone. We also find the mixture of deep features and chemical features more informative in terms of Shapley-based explanation values. Overall, our method is promising with its high accuracy rates and strong predictive skills. In addition to its predictive performance, DeepTHybrid provides a fast and scalable computational screening framework that can substantially reduce the cost and time associated with biological experiments by prioritizing high-confidence tumor T cell antigen candidates for downstream experimental validation.

Research into the evolution of biological networks enhances our understanding of the functional roles of various biomolecular properties. Graph growth models, such as the Preferential Attachment (PA) model, help characterize the evolutionary dynamics of protein interaction networks by modeling the preferential attachment of duplicated proteins' new interactions to existing ones. This approach generates realistic, scale-free PPI networks in a systematic manner. However, existing methods for reconstructing ancestral graphs based on the PA model have predominantly relied on greedy algorithms, often leading to suboptimal results. In this study, we introduce ILP-PA, a novel approach based on Integer Linear Programming (ILP), designed to reconstruct historical PPI graphs by maximizing likelihood within the Preferential Attachment model. Our ILP formulation also leverages systematic heuristics from general-purpose ILP solvers, allowing the analysis of near-optimal and multiple optimal solutions, which can be valuable for various applications across different fields. We evaluate the effectiveness of our approach on both synthetic data and three real protein-protein interaction graphs, specifically the Commander complex, bZIP transcription factor family, and herpesvirus interaction network. Compared to existing techniques, our ILP-PA solutions achieve higher likelihoods and demonstrate greater robustness to model mismatches and noise in the data. Furthermore, our solutions align more closely with biological findings from various studies across real datasets.

Topological domains (TADs) are consecutive genomic locations with denser local interactions to a certain extent, and they are important for cellular gene expression control and modulation. TADs were first identified when studying three-dimensional genomic structures over Hi-C interaction datasets. Many studies have focused on developing approaches in inferring TADs, which led to multiple TAD-caller approaches development. On the other hand, the number of RNA interactome datasets, such as RNA-RNA and RNA-DNA interactions, has recently been increasing. Even though TADs have been extensively studied in Hi-C datasets, they have not been studied across these RNA interactomes. We conducted a systematized comparison of 28 TAD-callers across mammalians over RNA-associated interactomes (RAIs), especially RNA-RNA and RNA-DNA interaction datasets at a high resolution. Our findings highlighted the significant enrichment of Cohesin/CTCF proteins at RNA-TAD boundaries for both RNA-RNA and RNA-DNA interactomes, especially those with corner dots, which are similar to the results for original TADs. The sizes and numbers of RNA-TADs vary significantly between different TAD caller approaches and RNA interactome resolution, suggesting the importance of considering RNA-TADs as hierarchical domains rather than distinct intervals. We examined the core principles and assumptions behind TAD-callers over RNA interactomes. To our best knowledge, this is the first time that many TAD inference methods are adapted to infer TAD-like domains on RNAs. Our results provide valuable guidance in selecting the most suitable methods for TAD inference over RNA interactomes.

Hi-C is an experimental technique to measure the genome-wide topological dynamics and three-dimensional (3D) shape of chromosomes indirectly via counting the number of interactions between distinct sets of loci. One can estimate the 3D shape of a chromosome over these indirect interaction datasets. Here, we come up with graph attention and residual network-based GAT-HiC to predict three-dimensional chromosome structure from Hi-C interactions. GAT-HiC is distinct from the existing 3D chromosome shape prediction approaches in a way that it can generalize to data that is different than train data. So, we can train GAT-HiC on one type of Hi-C interaction matrix and infer on a completely dissimilar interaction matrix. GAT-HiC combines the unsupervised vertex embedding method Node2vec with an attention-based graph neural network when predicting each genomic loci's three-dimensional coordinates from Hi-C interaction matrix. We test the performance of our method across multiple Hi-C interaction datasets, where a trained model can be generalized across distinct cell populations, distinct restriction enzymes, and distinct Hi-C resolutions over human and mouse. GAT-HiC can reconstruct accurately in all these scenarios. Our method outperforms the existing approaches in terms of the accuracy of three-dimensional chromosome shape inference over interaction datasets.

Changes in the prices of multiple financial assets over time can be characterized by their complex nature and interdependence. More traditional forecasting approaches may overlook the interdependencies among these assets, since they may not fully consider the spatial-temporal dependencies between them. Graph neural networks (GNNs) have emerged as powerful tools for modeling complex relational dependencies in areas such as social network analysis and traffic forecasting. However, their application in asset price prediction remains relatively unexplored. Here, we investigate GNNs’ effectiveness in forecasting multiple financial asset prices jointly, specifically in the foreign exchange (Forex) and cryptocurrency markets. We employ three spatio-temporal GNN frameworks-MTGNN, StemGNN, and FourierGNN-which are all recognized for their state-of-the-art performance in forecasting multivariate time series. These models transform time-series data into graphs and capture both spatial and temporal dependencies. They significantly outperform the baseline methods, including LSTM, ARIMA, and VAR, in predicting financial asset prices in the highly volatile cryptocurrency market. While the performance gap is less obvious in the relatively stable Forex market, GNN-based models still demonstrate a general advantage over LSTM, although they are outperformed by ARIMA. Through a series of experiments and backtesting strategies, we assess the predictive power and profitability of these models in portfolio construction. Our code and datasets are publicly available at https://github.com/seferlab/temporal gnn .

Identifying fraudulent financial reports and elucidating the mechanisms of fraud are critical for safeguarding investors from substantial losses. Financial statements present detailed accounting entries in tabular form; they inherently combine categorical and numerical variables governed by accounting dependencies, yet most existing methods fail to model interpretable interactions between these feature types. In this case, handling categorical variables together with numerical variables is important in enhancing the financial statement fraud detection performance. Here, we compare the methods for transforming categorical to numerical attributes, which are then used for financial statement fraud detection. We perform comprehensive experiments on two real-world datasets: FiGraph and USFSD. We compare 4 state-of-the-art specialized categorical-to-numerical transformation techniques with several other simpler statistical encoding mechanisms, such as target, label, Helmert, and GLMM encodings, as well as methods that can directly work on categorical data, such as CatBoost. These specialized transformation techniques are Hierarchical Coupling Learning-based CURE, Graph-based Categorical Embedding GCE, and Transitive Distance Learning-based embedding. The results reveal that the performance of CURE and XGBoost together surpasses all state-of-the-art techniques, achieving significant relative gains in macro-level recall over the second-best performing approaches, CatBoost and FT-Transformer, while also providing clear and interpretable insights into the discovered fraud pathways.

Background: Graph neural networks’ (GNNs) explainability, especially the explanation of edges and interactions among vertices in GNNs, is demanding mainly owing to dynamics and groupings between vertices. The existing graph explainability methods ignore the analysis of the following tasks weights over subgraphs but instead analyze solely sample-level explainability. Such sample-level explainability decreases their generalizability since it directly searches the explaining behaviour in the input dataset. Objective: In this study, we come up with a novel Orbit-based GNN explainer (OExplainer), which integrates both sample-level and method-level approaches over a predetermined set of subgraphs. As part of such analysis of subgraphs, our goal is to interpret graphs more comprehensively and intelligibly while providing each vertex’s explainability score for a particular graph instance. Methods: Our OExplainer decomposes the following graph neural network weights into explaining subgraph bases while identifying and characterizing particular predictions. By such characterization, we can carefully and accurately interpret the predetermined graph orbit’s role in vertex representation determination. In this characterization, we can also clarify the method’s behaviour generally for the whole input dataset. Moreover, we come up with novel vertex-specific scores in our subgraphbased approach over nonisomorphic graphlets. Such vertex-specific score encourages sample-level vertex improvement, and such improvement is related to the graph neural network’s vertex classification task. Results: Our experiments over simulated datasets confirm the importance and criticality of method weights in vertex classification explanation. In this case, method weight decomposition also has criticality. Our detailed experiments over multiple real protein-protein interaction datasets and metabolic interaction networks also exhibit enhanced performance in vertex classification. Conclusion: In both simulated and biological protein-protein interaction datasets, our approach outperforms the competing explanation approaches

Anti-money laundering is an international web of laws, regulations, and procedures aimed at uncovering money that has been disguised as legitimate income. Strict anti-money laundering (AML) laws and procedures require major and continuous transaction observation in inferring possible illegal events. Nevertheless, traditional rule-based approaches in banks frequently generate a significant number of false positives, which impose a major burden. In this case, deep learning approaches, especially graph-based Graph Neural Network-based (GNN) methods, could be explored in generating better anti-money laundering results. Here, we propose a diffusion-based AMLPD, which is novel in generating unsupervised node embeddings via learning graph embeddings inductively while detecting AML. AMLPD assumes a direction between edges, and it incorporates vertex and edge feature knowledge while encoding graph’s structure knowledge. AMLPD infers a vertex’s local state via combining diffusion with PageRank, which is an important knowledge for AML when embedded into low dimensional space Then, our approach can detect AMLs by a classifier using this low dimensional representation. Our approach can be scaled to larger data, as well as it can help with explainable AI by facilitating the embeddings analysis. According to experiments, our approach outperforms the baseline approaches. Therefore, AMLPD is favourable in enhancing the quality of GNN-based AML identification.

Pairs trading is a well-studied statistical arbitrage strategy including the identification of asset pairs exhibiting correlated changes in their historical prices. This statistical arbitrage strategy focuses on benefiting from non-permanent divergent behaviour of price, and it forecasts that the price relationship will revert to its usual and normal correlation. In this paper, we explore how more recent time-series-based deep learning techniques can be utilized in pairs trading, where cointegrated asset pairs are taken into account. We propose deep-learning and more traditional machine learning-based methods to predict the fluctuation of daily idiosyncratic residual terms between assets and their factor approximations. In our analysis, we focused on seven models: LSTM as a fundamental time-series method to capture interrelationships in dataset, Informer, Autoformer, iTransformer, Scaleformer, and Chronos as transformer-based deep time series methods, and AdaBoost, which is an ensemble learning-based machine learning method. We have assessed the performance of methods comprehensively over S&P 500 and cryptocurrency assets data starting from 2012 to 2020 respectively, and used a traditional statistical arbitrage-based relative value method as a baseline. All of our proposed learning-based methods turned out to be profitable strategies, obtaining higher Sharpe ratios and average returns by outperforming the baseline relative value method. Nevertheless, deep learning-based methods had a lower volume than the baseline, so when transaction costs are taken into account they showed better performance. Deep learning-based methods maximum drawdown was also lower than the traditional statistical arbitrage strategy. As a result, we show the benefits of time series-based deep learning methods in pairs trading across distinct asset classes.

Temporal gene expression is being analyzed via high-throughput profiling of molecular data over time. The expression values of genes are impacted by their previous expression values as well as the expression of interacting genes over time. Hi-C provides us with a broad genome-wide perspective on the interacting dynamics of genes. In this paper, we propose neural network-based spatio-temporal graph approaches STEPmr and STEPmi to predict changes in mRNA and miRNA expression over time, respectively. Both approaches can integrate a diverse set of Hi-C datasets and features obtained from Hi-C when predicting temporal expression patterns. Our methods can predict mRNA and miRNA expression with 77% and 93% correlation and ith mean squared errors of 0.21 and 0.01, explaining 59.1% and 88% of the variance, respectively. Important characteristics of the genes with the highest performances in both datasets are that they are structural signaling genes or transcriptional regulators involved in fundamental processes such as homeostasis, development, and RNA processing. Additionally, they are not limited to a specific cell type, but rather show constant expression throughout different tissues. In contrast, the lowest-performed genes generally behave in context-dependent expression patterns, where they include condition-specific biological functions instead of vital biological activities. These findings suggest a model of gene regulation and its predictability that is impacted by interacting gene dynamics. Our code and datasets are publicly available at . ### Competing Interest Statement The authors have declared no competing interest. The Scientific and Technological Research Council of Turkey (TÜB-TAK), 122E706

Gene expression and regulation with or without alternative splicing are key factors for cells to properly function. Distant splicing quantitative trait loci (distant sQTLs) are genomic mutations that impact the alternative splicing patterns of far-away genes. Nevertheless, the procedures causing a distant sQTL to regulate the alternative splicing of genes are not well defined. Higher resolution chromosome conformation capture experiments like Micro-C or Hi-C together with an expanding number of sQTL datasets on humans help us in understanding the spatial processes governing distant sQTL relationships at a genome-wide scale. In this study, we focus on analyzing whether spatial closeness helps in regulating sQTL-gene interactions over high-order chromatin topological domain structure, which is inferred from chromosome conformation experiments. We discover larger-scale chromatin shape to be in line with sQTL associations. In detail, sQTLs are generally spatially near their splicing genes in 3D, they frequently appear near topologically associating domain (TAD) and frequently interacting region (FIRE) boundaries, and are favorably related to genes over TADs and FIREs. Additionally, we discover that inside-domain sQTLs accompanied by functional regulatory elements, including enhancers and promoters, are spatially closer than all inside-domain sQTLs. This result suggests that spatial closeness between sQTLs and their distant splicing genes obtained from chromatin’s TAD structure has major importance in regulating alternative splicing and thus in gene regulation. Our results are robust across different experiments such as Hi-C and Micro-C, different TAD inference methods, different Hi-C binning resolutions, different alternative splicing events, and once we control for eQTLs, which are shown to be spatially close to their genes.

Predicting stock price movements can be framed as a classification task, where the goal is to anticipate whether a stock will increase, decrease, or remain stable. Most existing approaches rely solely on the movement patterns of individual stocks or stock pairs, overlooking the more complex, higher-order connections that exist among groups of stocks. In practice, stocks are often interrelated in higher orders, for example, by belonging to the same industry sector or being jointly held within the same investment fund. To address this, we compare 4 hypergraph neural network-based approaches to make spatio-temporal predictions for stock movement prediction, which explicitly leverages these higher-order dependencies. We use two heterogeneous hypergraphs, where one hypergraph represents sector-based associations and the other one represents fund-holding relationships among stocks. In general, we found the hierarchical hypergraph attention mechanism and temporal attention to be effective in achieving better performance. A hierarchical hypergraph attention mechanism models these relationships by weighting the contributions of stock nodes, hyperedges, and even the hypergraphs themselves. Temporal attention captures time-dependent dynamics of both stock and sector sequences, effectively accounting for the influence of past states. Experiments on real-world datasets demonstrate that the methods specializing in hypergraph integration achieve superior performance compared to existing methods, both in terms of predictive accuracy and profitability.

Chromosomes are intricate macromolecules composed of chromatin that create three-dimensional structures within cells. The arrangement and interactions of chromosomes play a crucial role in shaping the genome's structure, function, and gene expression regulation. To fully grasp the architecture and adaptability of the genome, it is essential to understand the complex relationship between chromosomal structure and nucleotide sequences. Advances in our understanding of genome topology have been significantly driven by next-generation sequencing technologies designed for capturing chromatin conformation. The exploration of 3D genome organization has led to the development of high-throughput chromatin conformation capture (Hi-C) techniques. Hi-C is essential for revealing new aspects of genome architecture and for mapping genome-wide chromosomal interactions, both within individual chromosomes and between them. These interactions include chromosomal territories, topologically associating domains (TADs), and chromatin or gene loops. This review provides a comprehensive overview of the historical development and current state of 'C' technologies, in-depth insights into various Hi-C techniques, and the analysis of 3D genome structures. It concludes with a discussion on computational tools for analyzing high-resolution Hi-C data, the challenges in modeling 3D genome structures, and potential future advancements in the field.

Decentralized and transparent nature of cryptocurrencies have lately increased investors interest in them. Forecasting cryptocurrency's price accurately is crucial to come up with a good investment strategy, and such a forecast requires one to consider its unique attributes as well as high volatility. Even though many existing studies have focused on analyzing the cryptocurrency transaction graph topology, studies on the analysis of transaction graph's impact on prices are quite limited. In this paper, we explore the forecasting ability of blockchain transaction graph-based attributes on Bitcoin's and Ethereum's future price via deep learning methods. More specifically, we came up with motif convolution module (MCM), a motif-based graph representation learning approach to take local structural knowledge into account more strongly in node and edge-attributed transaction graphs encoding substantial structural knowledge. Our proposed MCM constructs a motif dictionary without supervision, and employs a new motif convolution operation while extracting the vertices local structural context. Afterwards, we learn high-level vertex embeddings by using such structural context via multilayer perceptron and graph neural network. Overall, we extract the attributed transaction graphs temporally-evolving low-dimensional representations, and use such embedding data together with historical prices within self-attention-based LSTM to predict the future prices accurately. Our proposed approach outperforms all considered baselines in terms of both price and price direction prediction, showing the promise of efficient integration of transaction data into cryptocurrency price prediction.

Gene expression and regulation with or without alternative splicing are crucial for tissues and cells to function correctly. They have been studied from three almost independent perspectives at the genome level: 1- Recognition of splicing quantitative trait loci (sQTLs), 2- Expression quantitative trait loci (eQTLs) recognition, and 3- Recognition of longer-range physical chromatin interactions between genome segments which model 3D dynamics of cells and tissues. Even though the associations between eQTLs and longer-range chromatin interactions have been previously studied, a similar relationship between sQTLs and chromatin interactions has yet to be analyzed. In this case, examining whether sQTLs control the alternative splicing of their target genes’ mRNA via physically interacting genome segments is crucial. We have jointly analyzed high-throughput chromatin conformation capture (Hi-C) and sQTL datasets over eight human cancer tissues. We have discovered a positive association between the number of genes having sQTLs and chromatin interaction frequency. Such a positive association still exists when we also control for eQTLs. Additionally, sQTLs and their target genes generally exist inside identical topologically associating domains (TADs). These findings are observed over the whole set of analyzed cancer types and functional subsets of the sQTL dataset, such as survival-related sQTLs. Furthermore, tissue-specific sQTLs are statistically enriched in tissue-specific frequently interacting regions (FIREs) in 6 out of 8 human cancer tissues (Chronic Myeloid Leukemia, Colon Adenocarcinoma, Acute Myeloid Leukemia, Lung Adenocarcinoma, Prostate Cancer, Sarcoma). Our sQTL and Hi-C datasets have shown the existence of closer spatial distance between sQTLs and their target genes with possible alternative splicing across several different cancer types in humans. Such a closer spatial distance also exists, independent of whether we integrate eQTLs into the analysis. We found that sQTLs regulate alternative splicing through chromatin interactions. Source code of the analysis in this research is available on https://github.com/seferlab/sqtlhic .

Non-fungible tokens (NFTs) are unique digital assets whose possession is defined over a blockchain. NFTs can represent multiple distinct objects such as art, images, videos, etc. There was a recent surge of interest in trading them which makes them another type of alternative investment. The inherent volatility of NFT prices, attributed to factors such as over-speculation, liquidity constraints, rarity, and market volatility, presents challenges for accurate price predictions. For such analysis and forecasting, machine learning methods offer a robust solution framework.Here, we focus on three related prediction problems over NFTs: Predicting NFTs sale price, inferring whether a given NFT will participate in a secondary sale, and predicting NFT’s sale price change over time. We analyze and learn the visual characteristics of NFTs by deep pre-trained models and combine such visual knowledge with additional important non-visual attributes such as the sale history, seller’s and buyer’s centralities in the trading network, and collection’s resale probability. We categorize input NFTs into six categories based on their characteristics. Across detailed experiments, we found visual attributes obtained from deep pre-trained models to increase the prediction performance in all cases, and EfficientNet seems to perform the best. In general, CNN and XGBoost consistently outperformed the rest of them across all categories. We also publish our novel NFT dataset with temporal price knowledge, which is the first dataset to have NFT prices over time rather than at a single time point. Our code and NFT datasets are publicly available at https://github.com/seferlab/deep_nft.

Introduction More recent self-supervised deep language models, such as Bidirectional Encoder Representations from Transformers (BERT), have performed the best on some language tasks by contextualizing word embeddings for a better dynamic representation. Their protein-specific versions, such as ProtBERT, generated dynamic protein sequence embeddings, which resulted in better performance for several bioinformatics tasks. Besides, a number of different protein post-translational modifications are prominent in cellular tasks such as development and differentiation. The current biological experiments can detect these modifications, but within a longer duration and with a significant cost.Methods In this paper, to comprehend the accompanying biological processes concisely and more rapidly, we propose DEEPPTM to predict protein post-translational modification (PTM) sites from protein sequences more efficiently. Different than the current methods, DEEPPTM enhances the modification prediction performance by integrating specialized ProtBERT-based protein embeddings with attention-based vision transformers (ViT), and reveals the associations between different modification types and protein sequence content. Additionally, it can infer several different modifications over different species.Results Human and mouse ROC AUCs for predicting Succinylation modifications were 0.793 and 0.661 respectively, once 10-fold cross-validation is applied. Similarly, we have obtained 0.776, 0.764, and 0.734 ROC AUC scores on inferring ubiquitination, crotonylation, and glycation sites, respectively. According to detailed computational experiments, DEEPPTM lessens the time spent in laboratory experiments while outperforming the competing methods as well as baselines on inferring all 4 modification sites. In our case, attention-based deep learning methods such as vision transformers look more favorable to learning from ProtBERT features than more traditional deep learning and machine learning techniques.Conclusion Additionally, the protein-specific ProtBERT model is more effective than the original BERT embeddings for PTM prediction tasks. Our code and datasets can be found at https://github.com/seferlab/deepptm.