In this paper, we investigate knowledge forgetting in large language models with a focus on its generalisation—ensuring that models forget not only specific training samples but also related implicit knowledge. To this end, we begin by identifying a broader unlearning scope that includes both target data and logically associated samples, including rephrased, subject-replaced, relation-reversed, and one-hop reasoned data. We then conduct a rigorous evaluation of 15 state-of-the-art methods across three datasets, revealing that unlearned models still recall paraphrased answers and retain target facts in their intermediate layers. This motivates us to take a preliminary step toward more generalised implicit knowledge forgetting by proposing PERMU—a novel probability perturbation-based unlearning paradigm. PERMU simulates adversarial unlearning samples to eliminate fact-related tokens from the logit distribution, collectively reducing the probabilities of all answer-associated tokens. Experiments are conducted on a diverse range of datasets, including TOFU, Harry Potter, ZsRE, WMDP, and MUSE, using models ranging from 1.3B to 13B in scale. The results demonstrate that PERMU delivers up to a 50.40% improvement in unlearning vanilla target data while maintaining a 40.73% boost in forgetting implicit knowledge. Our code can be found in the supplementary material.

Modern networks require frequent control plane changes, which make it challenging to analyze the differences and similarities (i.e., relations) between network configurations before and after changes, as well as to identify the impacted specifications. Relational network verification effectively overcomes these problems. However, current tools still suffer from several limitations, including semi-automation, prolonged verification time, poor scalability, and limited expressiveness.To address these issues, this paper presents Relational Network Verification using Reinforcement Learning (RNV-RL). This framework integrates a specialized Domain-Specific Language (DSL) called Enhanced Regular Intermediate Relations (ERIR) with Reinforcement Learning (RL) techniques to automate relational network verification. RNV-RL translates input change intents and network configuration pairs into ERIR expressions, which compactly and comprehensively derive relational specifications from change intents, thereby enhancing enhance expressiveness. Building on this foundation, RNV-RL employs RL techniques to reduce verification time and improve scalability. It formulates the verification process as an optimization task for strategy search within a task-distribution Markov Decision Process (MDP) framework, where strategies are modeled as decision trees. A two-step approach is proposed to identify optimal strategies: the Multi-Distribution Policy Gradient (MD-PG) algorithm predicts the most promising strategies, and then the Policy-Guided Relational Network Verification Search (PG-RNVS) algorithm prioritizes and refines them.We compare RNV-RL with the latest relational network verification tool Rela and two single-snapshot verification tools, Batfish and Minesweeper. The evaluation results show that RNV-RL can express 100% of change intents while Rela only supports 97%. In contrast, Batfish and Minesweeper fail to extract change intents. Furthermore, RNV-RL is 10 times faster than Rela for 80% of the changes, while the other two tools are prone to timeouts. These performance gains are consistently observed across various specification sizes and location granularity levels.

Network configuration synthesis is essential for automated configuration management in large and complex networks. However, existing synthesizers face challenges in practical applications, including limited scalability, slow synthesis speed, insufficient support for various routing protocols, and difficulty in handling mixed vendor configurations. To address these issues, this paper proposes MoCS, a modular configuration synthesizer that integrates multiple Large Language Models (LLMs) with Graph Neural Network (GNN)-enhanced recommendations to enable protocol-agnostic and vendor-compliant configuration synthesis. MoCS decomposes the synthesis pipeline into three LLM-based modules, each following a unified prompt engineering framework with task-specific adaptations. Specifically, the Intent Translation Module (IT-Module) translates natural language intents into structured configuration tasks, while the Configuration Graph Generation Module (CG-Module) constructs a Configuration Knowledge Graph (CKG) by incorporating semantic information from network topologies, structured tasks, and vendor-specific configuration templates. These two modules collaborate to support various protocols and mixed vendor configurations via a unified graph representation. The Configuration Recommendation Module (CR-Module) utilizes a heterogeneous GNN-based model (HGAT-CR) to perform type-aware reasoning over the CKG and generate top-k candidate parameters. These candidates provide prior knowledge that narrows the search space and improves recommendation accuracy. Finally, they are refined through an LLM-guided optimization mechanism that combines formal verification feedback to produce the final configuration, ensuring maximal intent satisfaction while minimizing side effects. Our evaluation demonstrates that MoCS outperforms existing synthesizers, including NetComplete, INCS, and ConfigReco. In large networks with complex intents, MoCS achieves a high coverage rate (88.23 +/- 1.12%), low redundancy rate (7.89 +/- 1.59%), perfect intent satisfaction rate (1.00 +/- 0.00), and reasonable runtime (143.83 +/- 21.89s). Furthermore, MoCS can synthesize mixed vendor configurations, which current synthesizers cannot handle.

Cross-tokenizer knowledge distillation, where the teacher and student employ different tokenizers, is becoming increasingly prevalent, yet it poses underexplored challenges: existing methods fail to capture the rich knowledge encoded in teacher logits, as evidenced by the neglect of semantic information, inaccurate and biased logit alignment, and discarding distributional structure—ultimately leading to unfavorable distillation. To address these issues, we propose SeDi, a semantics and distribution-aware knowledge transfer framework tailored for cross-tokenizer distillation. To preserve factual knowledge, SeDi employs bipartite graph-based alignment at the tokenization level and a sliding window re-encoding strategy at the vocabulary level, enabling unbiased transfer of the teacher’s next-token predictions into the student’s vocabulary space. To further retain distributional information, we align the student’s entropy with that of the teacher by incorporating the student’s own logits during training, which helps to mitigate the exposure bias problem. Experiments on ten datasets across three task domains and five different teacher-student model pairs with varying vocabulary sizes demonstrate that SeDi delivers substantial improvements, with gains of up to 19.8%.

The goal of Session-based recommendation is to recommend the next item a user may be interested in based on historical click data, without relying on user profiles. Compared to the data from other recommendation scenarios, session data is typically more sparse, thus Self-supervised learning (SSL), which derives ground-truth from raw data, has gradually gained attention. Existing SSL methods usually augment data by dropping or transforming original sequences in Euclidean space, leading to two problems. Firstly, in Euclidean space, it is difficult to handle the distorted distribution and hierarchical nature of session data, and secondly, the essential spatial and temporal features of session data are usually overlooked. To address these issues, we propose the Hyperbolic Spatial-Temporal Network (HSTN), which enhances the performance by generating positive samples with the spatial-temporal features of session data in hyperbolic space. Specifically, we first use a hyperbolic hypergraph neural network as the base encoder to mitigate the influence of distorted data distribution. Then, we employ a spatial-temporal feature learning module to generate positive samples with spatial-temporal features for contrastive learning. Extensive experiments on three real-world datasets demonstrate that our proposed method achieves improvements of 11.6 %, 17.6 %, and 38.4 %, respectively, compared to current state-of-the-art methods, under P@10 metric.

Recent research indicates that adding residual connections in Graph Neural Networks (GNNs) would amplify susceptibility to anomalous nodes, consequently undermining the robustness of deep GNNs in practical settings. However, existing verification methods encounter challenges with the increasing number of parameters and computational overhead in deep GNNs. In this paper, we derive the general form of the residual connections and apply the dual back-propagation network to deep GNNs. Considering the heightened computational errors arising from the increased number of layers in deep GNNs, we propose a new method for calculating intermediate activation bounds of GNNs based on linear approximation. Experimental results show that new method can effectively enhance the verification accuracy. Notably, the maximum perturbation value of nodes correctly classified shows an average improvement of 119.5%. To showcase the the efficacy and scalability of our method, we verify robustness of deep GNNs on six different graph datasets, and our method can effectively verify the robustness of deep GNNs even with 32 layers of residual connections, i.e. verify over 87.29% of nodes in the Citeseer dataset. Furthermore, we analyse the influence of the graph structural properties on the robustness of the model.

Due to its powerful monitoring capabilities, In-band Network Telemetry (INT) technology has become essential to modern network management. However, a key challenge lies in reducing the redundant telemetry transmission from the data plane to the controller, a vital process for real-time network status monitoring swift decision-making. This challenge, if not properly addressed, could significantly waste network bandwidth and increase the controller's response time. In this paper, we present Cache-INT, a highly efficient in-network caching-enabled INT system. Specifically, Cache-INT stores and reuses network information collected by probes through a well-designed cache strategy, and uses the incremental transmission technique to reduce the volume of data transmitted. Furthermore, an INT system's performance depends heavily on probe path planning, which determines both the network coverage and control overhead. We optimize these factors in two scenarios: one with fixed cache-enabled routers, and the other allowing for flexible selection of cache-enabled routers. For the former, we use a Reinforcement Learning (DRL)-based algorithm to minimize control overhead and design a mask function speed up training. For the latter, we combine the simulated annealing algorithm and the DRL model to suitable cache-enabled router upgrade scheme, and then plan the probe paths accordingly. The experimental results show that, compared to existing INT systems, Cache-INT reduces the transmission volume from the data plane to the controller by at least 40% while achieving a 30% reduction in control overhead.

Due to its real-time visibility, In-band network telemetry (INT) is of great significance for network management. Nevertheless, with the rapid growth of network devices and services, targeted access to detailed network information in dynamic environments has become increasingly essential. This paper proposes an intelligent network telemetry system called NTP-INT to obtain more fine-grained network information on high-load switches. Specifically, NTP-INT consists of three modules: the network traffic prediction module, the topology pruning module, and the probe path planning module. Firstly, the network traffic prediction module adopts a Multi-Temporal Graph Neural Network (MTGNN) to predict future network traffic and identify high-load switches. Then, we design the topology pruning algorithm to generate a subnetwork covering all high-load switches to reduce the complexity of probe path planning. Finally, the probe path planning module uses an attention-mechanism-based Deep Reinforcement Learning (DRL) model to plan efficient probe paths in the subnetwork. Experimental results demonstrate that NTP-INT achieves more accurate telemetry on high-load switches while reducing control overhead by 50%. Additionally, the topology pruning strategy shortens training time by over 40%.

With the increasing complexity of networks, network telemetry becomes a critical part of network management. However, existing network telemetry systems still suffer from excessive control overhead, forwarding overhead, and latency. In this paper, we propose INT-LLPP, a novel in-band network-wide telemetry system with low-latency and low-overhead path planning. The network telemetry architecture of INT-LLPP is unique in that it only requires a set of probes to collect telemetry items for multiple service flows. Moreover, the proposed Probe Path Generation (PPG) algorithm optimizes the probe paths to reduce the forwarding overhead and achieve full network coverage. To balance the telemetry latency and control overhead, we propose an efficient algorithm called the Simulated Annealing Maximum Latency Setting (SAMLS) algorithm, which controls the length of the probe paths. Simulation results show that INT-LLPP can reduce network telemetry control overhead by over 50% and reduce forwarding overhead by 5% to 10%. Moreover, INT-LLPP can lower telemetry latency by 30% to 40%.

Deep neural network (DNN)-enabled edge intelligence has been widely adopted to support a variety of smart applications because of its ability to preserve privacy and conserve communication efficiency. The dilemma is that DNN models can be too large to be deployed on computationally constrained edge devices, and the volume of raw data can be too large to be efficiently transmitted to a centralized server. Thus, it is of utter importance that edge devices and cloud servers collaborate with each other to achieve fast and dependable model inference. Current collaborative solutions separate the DNN into two parts, which are placed and executed at the edge and in the cloud, respectively. However, these separated parts are executed consecutively, and all subsequent layers have to wait for the output of the previous layer even if they are not directly connected, causing significant inference latency. We propose a delay-aware edge-cloud collaboration (DECC) algorithm to reorganize the execution of DNN layers. By dividing DNN into several independent branches and selecting the optimal partition points, we apply a pipeline approach to parallelize the execution of these branches to minimize the inference delay. Extensive experiments show that the DECC outperforms existing methods by significantly reducing inference latency and improving throughput.

Machine learning-based services offer intelligent solutions with powerful models. To enhance model robustness, Federated Learning (FL) emerges as a promising collaborative learning paradigm, which iteratively trains a global model through parameter exchange among multiple clients based on their local data. Generally, the local data are heterogeneous, which slows down convergence. Knowledge distillation is an effective technique against data heterogeneity while existing works distill the ensemble knowledge from local models, ignoring the natural global knowledge from the aggregated model. This places limitations on their algorithms, such as the need for proxy data or the necessary exposure of local models to the server, which is prohibited in most privacy-preserving FL with a clueless server. In this work, we propose FedDGT, a novel knowledge distillation method for industrial server-clueless FL. FedDGT regards the aggregated model as the only one teacher to impart its global knowledge into a generator and then regularizes the drifted local models through the generator, overcoming previous limitations and providing better privacy and scalability support. Extensive experiments demonstrate that FedDGT can achieve highly-competitive model performance while greatly reducing the communication rounds in a server-clueless scenario.

Monitoring and anomaly detection of key performance indicators (KPIs) are crucial for large Internet companies to maintain the reliability of their Web services. Influenced by human behavior and schedules, the KPIs of Web services typically exhibit seasonal characteristics. These characteristics may be complex as different KPIs exhibit differences in trend, multiple periods, and noise behaviors. However, existing anomaly detection methods typically only model one fixed pattern of seasonal KPIs, which may lead to performance degradation when dealing with diverse seasonal KPIs. In this work, we propose a novel anomaly detection model for seasonal KPIs, Diner , which incorporates multiple interpretable components. It is able to capture the additive and multiplicative trends, multiple periods, and seasonal noise in intricate seasonal KPIs, making it easily adaptable to different types of seasonal KPIs. Additionally, we present a set of evaluation criteria for generic time series anomaly detection tasks, which prove more effective in handling ambiguous manual labels and various anomaly events. Experiments are conducted on three real-world datasets, and the performance Diner surpassed both the statistical baseline and the state-of-the-art deep learning baselines.

Changes in enterprise networks require updated configurations. However, manual configurations with slow update efficiency, poor performance, and handling limitations, lead to the unavailability of updated networks. Therefore, we propose an efficient network renascence framework, NetRen, which synthesizes OSPF/BGP configurations driven by service and traffic migration. We follow the workflow of sketch extraction, configuration synthesis, and repair. Initially, comprehensive graphs are constructed to represent configuration sketches. We propose a GraphTrans synthesizer with Transformer's benefits of long-range focus and parallel reasoning. Training samples with the optimization relationship enable the synthesizer to achieve a mapping that optimizes performance based on configurations. To overcome the satisfiability barrier, configurations from the synthesizer are input to the stepwise configuration repairer as well-initialized solutions, achieving rapid configuration repair. Experiments demonstrate that the consistency of network configurations output by the GraphTrans synthesizer averages 98%. NetRen achieves a 312.4x increase in synthesis efficiency and a 5.83% improvement in network performance.

Previous 3D hand pose estimation methods primarily rely on a single modality, either RGB or depth, and the comprehensive utilization of the dual modalities has not been extensively explored. RGB and depth data provide complementary information and thus can be fused to enhance the robustness of 3D hand pose estimation. However, there exist two problems for applying existing fusion methods in 3D hand pose estimation: redundancy of dense feature fusion and ambiguity of visual features. First, pixel-wise feature interactions introduce high computational costs and ineffective calculations of invalid pixels. Second, visual features suffer from ambiguity due to color and texture similarities, as well as depth holes and noise caused by frequent hand movements, which interferes with modeling cross-modal correlations. In this paper, we propose Keypoint-Fusion for RGB-D based 3D hand pose estimation, which leverages the unique advantages of dual modalities to mutually eliminate the feature ambiguity, and performs cross-modal feature fusion in a more efficient way. Specifically, we focus cross-modal fusion on sparse yet informative spatial regions (i.e. keypoints). Meanwhile, by explicitly extracting relatively more reliable information as disambiguation evidence, depth modality provides 3D geometric information for RGB feature pixels, and RGB modality complements the precise edge information lost due to the depth noise. Keypoint-Fusion achieves state-of-the-art performance on two challenging hand datasets, significantly decreasing the error compared with previous single-modal methods.

Pre-trained foundation models, particularly large language models, have achieved remarkable success and led to massive fine-tuned variants. These models are commonly fine-tuned locally and then uploaded by users to cloud platforms such as HuggingFace for secure storage. However, the huge model number and their billion-level parameters impose heavy storage overhead for cloud with limited resources. Our empirical and theoretical analysis reveals that most fine-tuned models in cloud have a small difference (delta) from their pre-trained models. To this end, we propose a novel lossless compression scheme FM-Delta specifically for storing massive fine-tuned models in cloud. FM-Delta maps fine-tuned and pre-trained model parameters into integers with the same bits, and entropy codes their integer delta. In this way, cloud only needs to store one uncompressed pre-trained model and other compressed fine-tuned models. Extensive experiments have demonstrated that FM-Delta efficiently reduces cloud storage consumption for massive fine-tuned models by an average of around 50% with only negligible additional time in most end-to-end cases. For example, on up to 10 fine-tuned models in the GPT-NeoX-20B family, FM-Delta reduces the original storage requirement from 423GB to 205GB, significantly saving cloud storage costs.

Software logs play a vital role in ensuring the reliability and availability of large-scale software systems. In recent years, researchers have made significant efforts to build log analysis approaches to manage software systems. However, these approaches focus on log compression, log parsing and log anomaly detection. In the current context, engineers continue to spend substantial time and effort on resolving errors once anomalous logs have been detected. To achieve truly automated software system management and high-level Artificial Intelligence for IT Operations (AIOps), it's necessary to bridge the gap between anomalous logs and their resolutions. In this paper, we propose a novel framework LogExpert to automatically generate recommended resolutions for anomalous logs. Specifically, we build a log recognizer to utilize the wealth of software knowledge in technical forums such as Stack Overflow (SO). In addition, LogExpert combines the great power of a Large Language Model (LLM) with domain-specific knowledge to generate the resolution. We conducted a preliminary evaluation of our framework on datasets from SO. Our log recognizer achieves the F1 score of 0.936. Our lexical metrics and human evaluation show the overall LogExpert framework achieves excellent performance in log-based resolution generation.

Anomaly detection in multivariate time series (MTS) is crucial for various applications in data mining and industry. Current industrial methods typically approach anomaly detection as an unsupervised learning task, aiming to identify deviations by estimating the normal distribution in noisy, label-free datasets. These methods increasingly incorporate interdependencies between channels through graph structures to enhance accuracy. However, the role of interdependencies is more critical than previously understood, as shifts in interdependencies between MTS channels from normal to anomalous data are significant. This observation suggests that anomalies could be detected by changes in these interdependency graph series. To capitalize on this insight, we introduce MADGA (MTS Anomaly Detection via Graph Alignment), which redefines anomaly detection as a graph alignment (GA) problem that explicitly utilizes interdependencies for anomaly detection. MADGA dynamically transforms subsequences into graphs to capture the evolving interdependencies, and Graph alignment is performed between these graphs, optimizing an alignment plan that minimizes cost, effectively minimizing the distance for normal data and maximizing it for anomalous data. Uniquely, our GA approach involves explicit alignment of both nodes and edges, employing Wasserstein distance for nodes and Gromov-Wasserstein distance for edges. To our knowledge, this is the first application of GA to MTS anomaly detection that explicitly leverages interdependency for this purpose. Extensive experiments on diverse real-world datasets validate the effectiveness of MADGA, demonstrating its capability to detect anomalies and differentiate interdependencies, consistently achieving state-of-the-art across various scenarios.

Nowadays, the rise of various network applications makes network traffic become increasingly complex, which brings more stringent requirements to traffic engineering (TE). Although the state-of-the-art TE approaches based on deep reinforcement learning (DRL) or traditional methods can generate optimal solutions for fixed traffic matrices, they cannot converge fast enough to provide real-time optimization in real networks either because of excessive computation times or high communication overheads. Moreover, due to the dynamically changing traffic load on the network, it is also challenging to achieve optimization of maximum link utilization (MLU) and end-to-end delay at the same time since these two optimization objectives may be conflicting, especially when the network is under a low traffic load, which makes the modeling very difficult. To meet these challenges, we present RT-TE, a TE system based on DRL and distributed message-passing between intelligent agents that can achieve real-time optimization for both MLU and end-to-end delay. To reduce the communication time due to link propagation delay during the optimization process, we design a proactive message-passing mechanism that allows agents to use partial messages to compute the routing policy while maintaining the optimization performance. Additionally, to achieve the tradeoff between the two optimization objectives, we model the propagation delay into the DRL model and design a multi-objective training framework with parameter transfer for training. Based on theoretical modeling, we can find the best tradeoff between the two objectives. Moreover, to improve the model's generalization for various traffic flows, we use a GNN model to generate the rewards of the DRL model, which greatly speeds up the training phase and allows us to feed massive amounts of data into the model. Through evaluations of real-world network topologies, our approach shows a 10%-20% improvement in optimizing MLU under short traffic-changing intervals and yields a 9%-13% improvement in optimizing end-to-end delay compared to state-of-the-art approaches.

As media streaming grows, service providers have to focus more on users' quality of experience (QoE). Among the metrics that influence QoE, network-oriented and media-oriented factors have been the primary concern of most existing work. Apart from the two factors, however, another crucial aspect of QoE is the user . Though the analysis and modeling of users is a turning point in the transition from QoS to QoE, the impact of users' memory and cognitive-psychological effects have not been fully explored. In this work, we analyze, validate and quantify the relationship between multiple cognitive effects and the QoE of users in media streaming. We propose PsyQoE, a new QoE assessment framework that uniquely focuses on user perceptions during media sessions. Besides providing real-time QoE assessments through leveraging machine learning, PsyQoE also predicts the overall QoE by taking into account user biases and long-term effects. Additionally, we have designed a further modified version to cater to the needs of hardware and energy-restricted scenarios. When compared with other existing methods, our approach demonstrates enhanced performance by boosting accuracy by 7% to 16%. It also provides better interpretability and broader applicability.