Power transmission network plays a crucial role in maintaining the stability and reliability of electrical systems. Fault detection and predictive maintenance are essential to ensure continuous operation and minimize downtimes, but traditional fault detection methods face challenges, particularly in remote areas where manual inspections are impractical. This paper presents a framework to enhance the steadiness and efficacy of power transmission networks through advanced fault detection and predictive maintenance. The proposed framework begins with data collection from power transmission sensors, including voltage, current, and temperature readings, along with historical fault records. Next, data pre-processing is performed using median imputation to handle missing values and categorical encoding to transform non-numeric data into numerical form. Feature extraction follows, where time-domain features like Peak-to-Peak Value, RMS, and Zero-Crossing Rate are computed to detect potential faults. The CatBoost model is then trained on the extracted features, and hyperparameter optimization is conducted using the Coati Optimization Algorithm. Once trained, the model performs fault detection and prediction, identifying faults such as Transformer Failures, Overheating, and Line Breakages. The model is assessed using metrics like accuracy of 99.42%, precision of 99.37%, recall of 99.40%, and F1-score of 99.38%. The framework achieves high performance in detecting faults and can be deployed in power transmission systems for proactive maintenance, reducing reliance on manual inspections, improving system reliability, and addressing challenges in remote locations.
Text summarization systems often struggle with selecting salient content, avoiding repetition, and handling out-of-vocabulary entities. We address these issues with a two-stage approach: a supervised sentence-ranking head (SRM-head) first selects the top-N sentences, and a Transformer generator then produces the summary. The generator is augmented with a time penalty in encoder-decoder attention to discourage reattending to recently focused source positions, and with a pointer mechanism that copies salient spans, thereby improving entity and number fidelity. Experiments on CNN/DailyMail and WikiHow, plus an additional evaluation on XSum, show that our model attains competitive ROUGE scores against recent pretrained systems while using lightweight, modular components.
With the rapid development of maritime commerce, the complexity of ship operation environments continues to grow, so that it is crutial for ship designers to make accurate forecasting of ship performance under various sea conditions. However, there exist the following challenges in ship performance evaluation and optimization. The first challenge is the computational overhead and slow response of numerical simulations based on Computational Fluid Dynamics (CFD), and the second is the inability of traditional machine learning to achieve the computational accuracy of numerical simulations. To address these issues, this paper proposes an online surrogate model, called Increformer, for ship performance prediction. The Increformer leverages continuous self-attention mechanisms to explore the temporal dependencies between feature variables and employs continuous normalization mechanisms to handle non-stationary data issues. In addition, in order to improve prediction accuracy by the model, we employ an incremental training strategy based on elastic weight consolidation to acquire new knowledge from data streams. Experiments are conducted by using historical performance data from various types of vessels including KCS, Wigley-III, and C60. The results demonstrate that the Increformer model effectively captures the temporal information and inter-dimensional correlations in the data, with the accuracy of ship performance prediction enhanced significantly. Furthermore, ablation experiments are also carried out to assess the effectiveness and necessity of the continuous normalization mechanism, continuous attention mechanism, and incremental training strategy for the Increformer model. The findings validate the accuracy and universality of the proposed model. It is also shown that the Increformer model adeptly captures trends and fluctuations in ship sequence data and thus provides a reliable solution for ship performance prediction.
Aspect-based dialogue sentiment quadruple analysis (DiaASQ) is a critical task in sentiment analysis, aiming to extract sentiment quadruples (target, aspect, opinion, sentiment polarity) from dialogues. Existing methods primarily focus on single-sentence sentiment analysis, often neglecting the rich contextual information and long-range dependencies in multi-turn dialogues. To address this limitation, we propose a novel memory framework, Memory, which incorporates adaptive contextual memory mechanisms to simulate human-like emotional refinement during conversations. Our framework consists of three key components: a Contextual Knowledge Memorizer to capture token-level syntactic-semantic dependencies, an Utterance-level Sentiment Interactor to model speaker-respondent dynamics, and a Multi-granularity Memory Integrator to fuse token-level and utterance-level information for precise sentiment relationship extraction. Extensive experiments on two benchmark datasets demonstrate the framework’s superiority, achieving 10.14% and 6.03% improvements in Micro-F1, and 13.07% and 5.60% improvements in Iden-F1 on Chinese and English datasets, respectively.
The core challenge of CTR prediction lies in how to effectively model feature semantics and their complex nonlinear interactions from high-dimensional sparse data. Existing methods mostly adopt static embedding in feature representation, ignoring the contextual dependency of semantics. In interaction modeling, traditional MLP suffers from gradient forgetting and low parameter efficiency, and are difficult to efficiently fit complex patterns such as non monotonicity. In response to the aforementioned limitations, this study proposes EFMNet, a hybrid architecture that integrates dynamic representation learning and frequency domain function approximation. Firstly, a feature dynamic enhancement module is designed, which generates context-aware complementary features through FRNet to reconstruct semantics, and uses SENet to evaluate the contribution at the channel level. Through affine transformation, it generates sample adaptive dynamic embedding. On this basis, an interaction paradigm of implicit and explicit collaboration is constructed: Fourier KAN is introduced as the implicit backbone, and its learnable edge functions based on Fourier basis are utilized to fit the nonlinear dependencies among features with high parameter efficiency; at the same time, CrossNet is integrated to explicitly model higher-order cross correlations, ensuring the effective transmission of key combined signals. Experiments on the Criteo and Avazu benchmarks show that EFMNet performs obviously better than strong baselines such as EulerNet and FinalMLP (e.g Criteo_AUC: 0.8164, Criteo_Logloss: 0.4427, Avazu_AUC: 0.7881, Avazu_Logloss: 0.3684), and its efficiency is comparable to that of the widely deployed DeepFM in the industry. This work validates the effectiveness of dynamic feature modeling and frequency domain approximation in CTR prediction, providing new ideas for recommendation system design.
Algerian Arabic (Darija) dominates digital communication in North Africa yet remains severely under-resourced in Natural Language Processing (NLP), hindering the development of robust applications for social media analysis and e-commerce. This paper addresses this scarcity by presenting a systematic framework for constructing and benchmarking Named Entity Recognition (NER) and Entity Linking (EL) resources tailored to the dialect’s linguistic complexity. We introduce a large-scale, multi-script dataset constructed through a novel hybrid methodology that integrates manual annotation of authentic texts, automated knowledge graph extraction from Wikidata, and rule-based synthetic generation. This approach ensures diverse coverage across ten semantic categories while explicitly addressing the challenges of code-switching and orthographic variation (Arabizi and Arabic script). A transformer-based model (XLM-RoBERTa) fine-tuned on this resource achieves state-of-the-art performance, demonstrating significant robustness compared to existing baselines. Beyond the dataset, we provide a practical deployment interface and comprehensive evaluation metrics, establishing a crucial foundation for advancing NLP capabilities in North African dialects and facilitating downstream tasks such as content moderation and cultural heritage preservation.
Semantic segmentation in urban scenes is an important task in computer vision. However, urban road scenes still present many challenges, such as category imbalance and complex backgrounds. These problems lead to unclear edge segmentation and inaccurate classification of occluded objects in existing semantic segmentation methods for urban scenes, which limits their accuracy and robustness in practical applications. In this paper, we propose a model that recursively enhances edge feature representation while incorporating local spatial context. To address the problem of unclear edge segmentation, we introduce Multi-scale Central Difference Convolution (MS-CDC) to fuse multi-scale edge features. The feature pyramid-based FeedBack Connection (FBC) module fuses multi-scale features while recursively enhancing the original network, thereby improving the robustness of the model to occluded objects. Meanwhile, we design a Local Feature Extraction (LFE) module to capture pixel-wise relationships by constructing local pixel graphs and center pixel graphs. It can learn local contextual information to extract finer-grained pixel features. Experimental results on the Cityscapes and Mapillary Vista datasets validate the effectiveness of the proposed model. Our model achieves 80.67% and 45.5% mIoU on the validation sets of Cityscapes and Mapillary Vista, respectively. We open-source our code at https://github.com/sanmanaa/segmentation-autodriving-graph-centralconv .
Stock price prediction remains a challenging yet critical task in financial research, with significant implications for both investors and policymakers. However, due to the combined influence of external factors (e.g., political and market conditions) and internal factors (e.g., managerial competence and organizational structure), stock prices exhibit high volatility. In prior studies, recurrent neural network (RNN)-based deep learning models have dominated, where stock price data is modeled as a time series. Most researchers have focused on single time-scale features for model training, failing to capture stock price fluctuations across multiple time scales, ultimately leading to suboptimal performance in highly volatile periods. To address this issue, we propose a time attention-driven long short-term memory (TAD-LSTM) network combined with a dynamic balanced scalable sparse variational Gaussian process (DBS-SVGP) auxiliary learning method for stock price prediction. The TAD-LSTM module extracts coupling relationships between temporal patterns, enabling efficient feature extraction and fusion. Meanwhile, the DBS-SVGP module leverages temporal and feature variables to enhance TAD-LSTM outputs, enabling precise prediction across different time-scale fluctuations while overcoming scalability limitations in large datasets. Furthermore, the proposed approach integrates symmetry principles, ensuring robust predictive performance under dynamic market conditions. We conducted extensive experiments on the CSI300 Index Dataset, Alibaba Stock Dataset, Google Stock Market Dataset, and Netflix Stock Price Dataset to evaluate our proposed approach, systematically comparing it against state-of-the-art stock price prediction models. Experimental results demonstrate that TLD-SVGP outperforms all competing models across all four datasets, achieving the highest average R 2 value (96.26%) and the lowest average MAPE (4.59%).
Consistent use of personal protective equipment (PPE) reduces exposure to industrial hazards, yet continuous compliance monitoring remains difficult in fast-moving workplaces. This study evaluates lightweight, real-time PPE detectors using two purpose-built datasets: PPEDS-1000 (1,000 manually annotated real-world images) and PPEDS-2600 (an augmented extension designed to reflect deployment variability). YOLOv13-nano and YOLOv13-small, implemented from the publicly available iMoonLab repository, were trained as untuned baselines, while optimization focused on YOLOv8 due to its mature tooling and scalable lightweight variants. Optuna-based hyperparameter optimization was restricted to YOLOv8-nano and YOLOv8-small to preserve deployment feasibility, tuning key training factors such as learning rate, batch size, and weight decay with validation mAP50 as the objective. Compared with pre-optimization baselines, the single-run best configurations achieved absolute gains of up to 2.5 percentage points in mAP50 and 3.2 percentage points in mAP50-95. However, an exploratory repeated-run analysis suggests that the stability of these gains is dataset-dependent. On the smaller PPEDS-1000 dataset, tuning benefits appeared seed-sensitive, whereas on the larger and augmented PPEDS-2600 dataset, tuning showed more consistent behavior together with lower variance across random initializations. Overall, the findings indicate that structured hyperparameter tuning is promising for deployment-oriented PPE monitoring, but its ability to yield stable gains appears to depend on adequate dataset scale and representative variability.
Abbreviation disambiguation (AD) aims to select the most appropriate definition from a set of candidates. Large language models (LLMs) can perform various tasks without specific fine-tuning by using demonstration examples of in-context learning (ICL). However, significant challenges lie in harnessing the capabilities of LLMs for the AD task by designing effective ICL techniques. In this paper, we propose an IN-context-Learning-based AbbreviatioNDisambiguation framework (IN-LAND). IN-LAND introduces a novel mechanism for selecting demonstration examples in AD tasks, evaluating examples based on three criteria: strict abbreviation similarity, loose abbreviation similarity, and sentence semantic similarity. By selecting optimal training samples, this framework aims to improve the inference capabilities of LLMs. Our experimental results demonstrate the efficacy of IN-LAND in French, Legal English, and Malay. Compared to the Random K-shot ICL baseline, our method achieves consistent improvements across the average performance of four different LLMs. The impact is particularly notable in less commonly spoken languages. For instance, in Malay, our method improves the average macro F1 score and accuracy by 7.11% and 9.19%, respectively. In French, the increases are 2.69% for the macro F1 score and 3.17% for accuracy. Similarly, for Legal English, our approach enhances the average accuracy rate by 1.04% while maintaining competitive performance in macro F1 score. These results underscore the robust capabilities of LLMs and their ability to generalize effectively across various languages.
Recommending news articles to anonymous users is a common yet challenging task. In recent years, several studies have focused on personalized recommendation based on user multi-interests in anonymous session scenarios. However, these studies neglect to utilize important temporal information within sessions when modeling user multi-interests, such as user browsing duration and news publication time. To address the issue, we propose a time highlighted multi-interest network for session-based news recommendation (TMN-SNR), aiming to explore users' interests in various aspects of news and their degrees of interest by incorporating temporal information from news sessions, thereby comprehensively and precisely portraying user multi-interests. Specifically, we integrate the diverse content of news and news timeliness, considering the varying durations of users' news browsing and news publication times, to achieve a comprehensive representation of user multiple interests. We also conduct comparative analyses of the impact of different multi-interest recommendation strategies on news recommendation performance. Furthermore, we propose a negative sampling method that integrates news publication time proximity and content similarity to enhance the representativeness of negative samples and improve model training performance. Experiments based on three real-world datasets and comparisons with baseline model performances prove that our method exhibits higher accuracy.
Sentiment analysis is a fundamental task in Natural Language Processing (NLP) that aims to automatically identify opinions and emotions expressed in textual data such as customer reviews, social media posts, and online feedback. However, sentiment classification remains challenging due to issues such as multilingual content, rating bias, informal language, and inconsistent relationships between review text and star ratings. For example, a review may contain positive text but be associated with a low rating, or include mixed sentiments within the same sentence, making accurate classification difficult. To address these challenges, this study proposes the Mathematical Optimization-Based Sentiment Tagging (MOST) framework, which integrates transformer-based language models with the Multi-Objective Optimization on the Basis of Ratio Analysis (MOORA) technique for robust sentiment classification. In the proposed approach, contextual sentiment representations are extracted using SBERT, T5, and XLNet, and their probability scores are organized into a decision matrix where sentiment classes act as alternatives and model outputs serve as evaluation criteria. The MOORA optimization method is then applied to aggregate these scores and determine the final sentiment label. The proposed framework aims to answer the following research questions: (i) how transformer-based feature extraction combined with optimization techniques can improve sentiment classification accuracy, (ii) whether the model can remain robust across multilingual datasets, and (iii) how rating-independent sentiment tagging can mitigate rating bias. Experimental evaluation on multiple English and Hindi datasets demonstrates that the MOST model achieves up to 92-93% accuracy, outperforming several existing approaches while maintaining strong cross-domain and cross-language performance. These results highlight the effectiveness of integrating deep contextual embeddings with mathematical optimization for reliable sentiment analysis.
As large language models (LLMs) evolve rapidly, distinguishing AI-generated text (AIGT) from human-written text (HWT) is becoming increasingly challenging. Recently, some AIGT detectors have been developed to overcome this challenge and have achieved decent accuracy. However, their brittle text representations make them highly susceptible to text perturbations, such that even minor character-level perturbations can reverse their predictions. In this work, we propose a multi-grained latent feature denoising and contrastive representation learning architecture to enhance text representations in terms of granularity, robustness, and distinguishability of features, thereby achieving robust AIGT detection. Specifically, we first extract both document-level and fine-grained segment-level features using a dual network, which captures the global and subtle local differences between AIGT and HWT. To encourage feature stability under perturbations, we inject random noise into both latent features and employ a denoising network to reconstruct the original representations. While this does not precisely simulate discrete character-level perturbations, it acts as a feature-level regularizer that suppresses non-essential variations and promotes smoother, more stable representations. Considering the similarities between AIGT and HWT, we further design a contrastive augmentation mechanism to increase the distinguishability between them. Extensive experiments demonstrate that our method not only outperforms baseline models in terms of classification accuracy but also exhibits superior robustness against various text perturbations.
The increasing availability of fine-grained student behavior data on smart campuses offers significant opportunities for personalized education. However, traditional clustering methods applied to such structured data often fail to capture the semantic complexity of behavioral features and the relational dependencies among individuals. To address these dual challenges, we propose a deep unsupervised clustering framework that integrates Bidirectional Encoder Representations from Transformers (BERT) and Graph Attention Networks (GAT). Recognizing that raw numerical features lack contextual depth, our approach first transforms structured data into natural language profiles, leveraging a pretrained BERT model to extract semantically rich embeddings. These individual representations are situated within a student behavior graph, where a GAT module refines node features by capturing relational structures and inter-student similarities. The combined embeddings enhance the performance of multiple clustering algorithms in identifying distinct behavioral patterns across students. In addition, we introduce a hierarchical anomaly detection module that identifies both unstable behavior clusters and outlier individuals based on intra-cluster variance and local density, providing a solution for detecting anomalous patterns in student populations. Experimental results on real-world campus datasets demonstrate the framework's effectiveness, while further analysis highlights its practical utility in uncovering early indicators of academic risk through interpretable behavioral modeling.
Transformer-based policy networks have shown promising prospects in Portfolio Selection (PS). However, constrained by high computational complexity, they primarily focus on short-term price series (e.g., the past 12 h), while overlooking valuable long-term features. Furthermore, the inherent volatility and noise tend to lead the model to overfit on low-level information. To tackle these challenges, we propose an innovative approach. This approach leverages a pre-training model to extract high-level patterns from extended price series (e.g., the past two weeks) and then enhances decision-making in policy networks. The pre-training model is designed based on discrete representations to achieve better generalization and interpretability. Specifically, we tokenize the price series into discrete tokens through Vector-Quantization Variational AutoEncoder (VQ-VAE) and encourage the pre-training model to reconstruct these tokens according to the masked price series. Then we introduce a compact encoder-only policy network named Portfolio Transformer Encoder (PTE). Finally, PTE is provided with high-level patterns from the pre-training model to make more comprehensive decisions. We term the whole approach as Vector-Quantization Portfolio Transformer Encoder (VQ-PTE). VQ-PTE demonstrates superior performance on real-world currency and S&P500 datasets, achieving a minimum improvement of 35% in returns. Additionally, visualization results highlight superior interpretability.
Knowledge graph-based question answering (KGQA) systems face several challenges. These include the need for detailed training data, difficulty in handling complex multi-hop queries, and dense knowledge gap interactions. The model needs training on annotated entities and relations, which requires significant human effort and time. We developed methodologies that improve end-to-end question answering with knowledge graphs, eliminating the need for pre-annotated entities (gold entities). Our approach incorporates language models-Text-to-Text Transformer (T5) and Longformer-and employs a named entity disambiguation technique. We reduced the dependency on gold entities by first removing explicit entity annotations from the training data and then augmenting this data with relevant knowledge base facts. In this paper, we explored two different methodologies: (1) training T5 and Longformer on this augmented dataset to answer factoid questions using inferred knowledge graph entities, and (2) applying transfer learning with SPARQL-based supervision to improve generalization. The experimental results demonstrate that the proposed models are efficient and offer effective strategies for addressing complex questions while significantly reducing the need for manual annotation of training data.
Aim COVID-19 (SARS-CoV-2), which was reported to be highly transmissible and to have a low case-fatality rate, has had a significant impact on the global public health system.Subject and Methods To provide accurate long-term spatiotemporal prognostics of future COVID-19 (SARS-CoV-2)-related infection and death rates, the authors benchmark state-of-the-art biosystem risk-evaluation methodologies, which are particularly suitable for multiregional environmental, biological, and public health systems. The main aim of this case study was to assess future coronavirus-related mortality rates for a specified return period within the relevant region. The authors applied a novel, population-based, multicentre, clinical-data-based statistical method directly to the raw clinical dataset. Extending Extreme Value Theory (EVT) and the Generalized Extreme Value (GEV) distribution from univariate (1D) to bivariate (2D) models presents specific challenges. 1D EVT/GEV cannot be readily extended to 2D, let alone to higher-dimensional biological systems. Epidemiological biosystem performance or limit state function depends on multiple random variables (e.g., covariates, resistance, bio-environmental factors). Engineering relevance: assessing the reliability of high-dimensional bio-systems where risks are low (e.g., less than 10-3).Results The proposed spatiotemporal method can be applied effectively across a wide range of national public health models, using raw data from national clinical surveys. Novelty: This case study benchmarks a recently developed multivariate bio-reliability method, utilizing a raw (unfiltered) clinical data sample. Primary novelty lies in the ability to treat virtually infinite-dimensional biosystems without any prior knowledge of the underlying joint distribution, as required by classic methods like FORM and SORM (First and Second Order Reliability Methods, respectively).Results The proposed spatiotemporal method can be applied effectively across a wide range of national public health models, using raw data from national clinical surveys. Novelty: This case study benchmarks a recently developed multivariate bio-reliability method, utilizing a raw (unfiltered) clinical data sample. Primary novelty lies in the ability to treat virtually infinite-dimensional biosystems without any prior knowledge of the underlying joint distribution, as required by classic methods like FORM and SORM (First and Second Order Reliability Methods, respectively).Conclusion The underlying issues in clinical dataset quality analysis are briefly discussed, along with significant methodological limitations. Plane language summary State-of-the-art public health system bio-reliability and risk evaluation method was benchmarked utilizing the Coronavirus Disease 2019 (COVID-19) raw clinical dataset. Confidence ranges have been forecasted for predicted epidemiological risk levels. Accurate multi-modal risk forecasts were provided.Conclusion The underlying issues in clinical dataset quality analysis are briefly discussed, along with significant methodological limitations. Plane language summary State-of-the-art public health system bio-reliability and risk evaluation method was benchmarked utilizing the Coronavirus Disease 2019 (COVID-19) raw clinical dataset. Confidence ranges have been forecasted for predicted epidemiological risk levels. Accurate multi-modal risk forecasts were provided.
Noisy labels are often present in large, accessible datasets. Learning with noisy labels can degrade the generalization performance of DNNs. While Semi-Supervised Learning (SSL) approaches have shown promise by predicting pseudo-labels to correct noisy labels, we find and demonstrate a fundamental limitation of SSL-based label correction methods: hard samples near decision boundaries significantly weaken the memorization effect that these methods rely on. This leads to erroneous pseudo-labels and creates a negative feedback loop where models gradually memorize these errors, further degrading performance. To overcome the issue, inspired by AdaBoost and building on these insights, we propose HEALON (Hard samplE Adaptive Labeling with Optimal reweighting for Noisy labels), a novel framework for learning with noisy labels that effectively addresses the hard sample challenge through an optimal weighting strategy that balances their influence during training. The framework primarily consists of two steps: Weighted Implicit Ensemble (WIE) and Weight Optimization for HArd Sample (WOHAS). WIE combines the Adaboost strategy with multiple sets of weight distributions obtained from WOHAS to train a single model, allowing the model to converge to multiple local optima along its optimization path and predict pseudo-labels. A sample difficulty minimization method is then designed to aggregate the predicted labels, generating near-global optimal pseudo-labels for each noisy sample, followed by a multiple "snapshot" weights strategy to minimize computational cost. WOHAS quantifies sample difficulty using information entropy and derives optimal weights to WIE via a cumulative sample difficulty strategy, balancing the impact of hard samples while preserving the memorization effect. Extensive experiments on benchmark datasets demonstrate that our approach significantly outperforms state-of-the-art methods in both pseudo-label correction accuracy and overall classification performance.
Objective: This study aims to improve Left Ventricle (LV) quantification accuracy and efficiency in cardiac disease diagnosis using automated image analysis.Methods: We propose XNet, a deep learning framework based on an extended convolutional network with multi-task learning. XNet segments LV structures estimates Regional Wall Thickness (RWT) and classifies cardiac phases (systole/diastole) from 2D Positron Emission Tomography (PET) sequences. The model integrates spatial and temporal features and is trained on augmented PET datasets.Results: XNet outperformed existing methods, achieving a mean absolute error of 1.5 mm, classification accuracy of 98.5%, validation accuracy of 97.2%, and a loss of 0.048. It showed strong performance in low-contrast and varied-quality image conditions, reducing myocardial area estimation error by 38.6% compared to baseline models.Conclusion: XNet provides a robust, accurate, and fully automated solution for LV quantification, offering a reliable tool to support clinical diagnosis and treatment planning in cardiovascular care.
Temporal knowledge graph completion aims to predict missing entities and relationships over time. However, traditional methods often fail to effectively capture temporal dynamics and frequency-domain features. They struggle to properly weigh historical information across varying time intervals and overlook periodic patterns inherent in the data. To address these shortcomings, we introduce TDFT, a novel model that integrates time decay factors and frequency-domain transformations. TDFT incorporates a time decay mechanism to prioritize relevant historical information and uses frequency-domain transformations to uncover latent periodic features. These features are learned via a frequency-domain neural network, improving the model's ability to model periodic fluctuations in entities and relationships. Experimental results show that TDFT outperforms existing methods on multiple benchmark datasets.
