Temporal Heterogeneous Networks (THNs) are evolving networks that characterize many real-world applications such as citation and events networks, recommender systems, and knowledge graphs. Forecasting THNs involves predicting future connections within a network that evolves over time and comprises diverse types of nodes and interactions with varying temporal dynamics. Although some Graph Neural Networks (GNNs) models have been successfully applied to forecast THNs, there is a lack of a general overview of how the message-passing computation could be extended to treat THNs. Moreover, most of the current solutions exhibit pitfalls in their training and evaluation strategies. Hence, in this work, we propose a graph deep learning framework for THN forecasting. Our framework decomposes the computation of a GNN layer into multiple components and introduces two different schemes to update embedding representations for THNs. This design allows the classification of existing solutions into special instances of our framework and highlights their potential limitations. We also extend the set of benchmarks for THNs by introducing two novel high-resolution temporal heterogeneous graph datasets derived from an emerging Web3 platform and a well-established e-commerce website. Overall, we conducted the first massive evaluation of THNs solutions over four temporal heterogeneous network datasets on two different future link prediction tasks using a fair newly introduced evaluation setting that considers the evolving nature of the data. Based on the limitations of existing solutions, we develop a new model that combines working techniques from previous models and leverages a new embedding update scheme. Experiments show the prediction power of our model compared to current solutions for link prediction in temporal graphs. Moreover, the experimental evaluation highlights the strengths and weaknesses of the different solutions and shows the effectiveness of our framework design.
As deep learning models grow in complexity and parameter, many researchers have attracted their attention to distributed training and hybrid parallel strategies. However, current methods are often based on dynamic programming, leading to a large time cost. There exist only a limited number of approaches to exploring deep learning techniques, and these approaches demand considerable time for online training. In this paper, we develop the Automatic 3D Parallelism Strategies Search (APSS), which is an efficient strategy search algorithm based on a pre-trained neural network and a heuristic policy. APSS initially employs a heuristic policy in conjunction with the training cluster environment to derive candidate strategies. Subsequently, a Deep Pipeline Strategy Network (DPSN) is proposed to provide a detailed pipeline partition for each candidate strategy. APSS utilizes an offline pre-training method called Contrastive Reinforcement Learning (CRL) to pre-train DPSN, eliminating the need for real data collection and fine-tuning during subsequent application. In search speed, APSS achieves enhancements by an order of magnitude faster than AMP, a prominent framework based on the open-source DeepSpeed. In terms of the quality of parallel strategies, APSS can still identify the optimal strategy. The source code will be made publicly available on GitHub1.
Graph Neural Networks (GNNs) have attracted significant attention for their effectiveness in fake news detection, particularly due to their capability to leverage the social context embedded within news dissemination. Most existing studies, however, operate on static heterogeneous information networks (HINs), assuming these graphs adequately capture the complex interactions among social entities. In practice, the dynamic nature of real-world social networks presents a substantial challenge, as training on suboptimal or outdated graph structures can severely limit the expressiveness of GNNs. Although various approaches have been proposed to model dynamic HINs, they often rely on computationally expensive message-passing mechanisms to update node embeddings, which hinders scalability to large social graphs. In this paper, we introduce DHGNN (Dynamic Heterogeneous Graph Neural Network), a novel model designed to address these challenges. DHGNN simplifies traditional GNN message-passing by employing a dynamic propagation scheme inspired by the personalized PageRank tracking process in HINs. Extensive experiments on three real-world benchmark datasets demonstrate the effectiveness and efficiency of DHGNN in detecting fake news.
While large language models (LLMs) have demonstrated impressive results on public benchmarks, their effectiveness in structured, real-world problems like behavioral analytics remains underexplored. This work assesses the out-of-the-box performance of LLMs for industry-specific downstream tasks, with player churn prediction as a representative task. Evaluating LLMs on public benchmarks risks data leakage and task-specific overfitting, so instead we perform experiments on a novel self-compiled dataset for churn prediction, a task not part of any standard benchmark. We compare the performance of OpenAI's GPT-4.1 with traditional machine learning models, such as XGBoost and MLPs, and analyze the impact of the LLM's extensive internal world knowledge and reasoning capabilities. With few-shot prompting, GPT-4.1 achieves a weighted F1 score of 0.787, matching the performance of XGBoost on the same set of samples. We show that the LLM can compensate for missing information with its internal world knowledge and reasoning capabilities, performing best if it can leverage both. Our results highlight the potential of LLMs for cross-game churn prediction and other structured, industry-specific tasks.
Large Language Models (LLMs) have demonstrated remarkable capabilities but their significant computational and memory demands hinder widespread deployment, especially on resource-constrained devices. Quantization, the process of reducing the numerical precision of model parameters, has emerged as a critical technique for compressing LLMs and accelerating inference. This paper provides an overview of LLM quantization, with a particular focus on the Post-Training Quantization (PTQ) methods implemented within the popular llama. cpp framework and its GGUF file format. We begin by covering quantization fundamentals, including the distinction between PTQ and Quantization-Aware Training (QAT). We then describe the specific PTQ schemes employed by llama. cpp, including legacy methods, advanced K-quants, and recent IQ-quants, along with their underlying mathematical principles. The paper also discusses the impact of these techniques on model fidelity, hardware requirements, inference speed, and traces the adoption of GGUF as a de facto standard in the open-source community. This work serves as a practical guide and comprehensive reference for researchers aiming to deploy LLMs on resource-constrained hardware. By systematically documenting and comparing the PTQ methods within llama. cpp, we provide the necessary insights to navigate the trade-offs between model fidelity, inference speed, and memory footprint. This enables informed decision-making for real-world applications, from local CPU-based inference to efficient edge deployment.
