Traffic engineering (TE) in the wide-area networks (WAN) is crucial for optimizing network performance by distributing traffic across various paths. Traditional methods use optimization like linear programs (LP) to solve TE, resulting in long decision-making time especially in larger topologies due to the iterative nature of optimization algorithms. Deep neural networks (DNNs) have been recently applied to TE to expedite decision-making through model inference. However, existing works rely on discrete event-based network simulators or non-differentiable algorithms to compute TE metrics from decisions by interacting with the network environment, breaking the gradient chains from decisions to metrics. Thus, TE metrics are treated as scalar values without gradients, limiting the learning paradigm for TE to reinforcement learning (RL) only. This paper demonstrates that calculating TE metrics can be fully differentiable, enabling direct gradient-based DNN updates for better TE decision-making, surpassing RL's reliance on approximated value functions. We propose dNE, a lightweight network simulator that uses differentiable matrix operations to evaluate TE decisions and compute user-defined metrics, enabling advanced DNN training paradigms like goal-driven optimization supervised by TE metric gradients. With dNE, experiments on four DNN-based TE algorithms show that DNNs trained with metric gradients reduce performance loss by over 10x compared to RL, speed up LP solvers by 13000x, and achieve 1000x faster metric computation than traditional event-based simulators.

Network traffic classification in open-world settings, where applications evolve dynamically and labeled data is limited, remains an important unsolved problem. Existing techniques designed for static closed-world scenarios struggle when faced with shifting traffic patterns and categories. In this work, we formulate open-world flow classification (OWFC) as a crossmodal translation problem and identify four key challenges: (1) bridging the modality gap between traffic data and class labels, (2) discovering novel unknown classes at run-time, (3) learning representations for complex traffic features, and (4) rapidly adapting to new environments. To address these challenges, we propose HeraClass, a novel OWFC system that models temporal and categorical traffic features separately and learns them automatically, and maps traffic to a semantic language space to align known flows and their class labels, while making unknown flows depart from known class labels and thus maintaining low confidence to identify them as known ones. HeraClass supports real-time classification and can efficiently adapt to new environments using low-rank adaptation. Testbed experiments demonstrate HeraClass’s strong performance on both standard benchmarks and open-world scenarios, outperforming state-ofthe-art baselines by $\mathbf{1 2 0. 1 0 \%}$ on average for OWFC. We opensource pre-trained models, datasets, and implementation code to facilitate future research1.

Border Gateway Protocol (BGP) security is critical to Internet infrastructure, yet progress in routing anomaly detection has been limited by the scarcity of publicly available incident datasets, which contain only 18 recorded cases. We observe that public operator mailing lists, e.g., NANOG and AusNOG, contain abundant yet largely untapped reports of real-world routing anomalies. To leverage this source, we develop an LLM-assisted extraction pipeline that identifies 244 candidate incidents from historical discussion threads. After expert validation, we curate a verified benchmark containing 232 confirmed routing anomaly events, making it 11.89X larger than existing dataset. Using this benchmark, we show that existing routing anomaly detection systems generalize poorly to diverse real-world incidents. At the same time, we find that some general-purpose LLMs without routing-specific adaptation can identify a subset of routing anomalies, but their performance varies across models and remains insufficient for reliable routing anomaly detection. Motivated by this observation, we design ROUTELLM, an LLM-based routing anomaly detector that integrates BGP-semantic-aware tokenization, routing-domain adaptation, and time-aware routing evidence retrieval. Experimental results show that ROUTELLM achieves 87.13

Efficient scheduling of service applications is critical for improving cluster resource utilization while minimizing makespan and application completion time. However, existing schedulers often struggle with coordinating task placement on worker machines due to the lack of runtime estimations. This limitation leads to two major performance issues: the non-synchronization problem and the contention-oblivious problem, both of which result in suboptimal application completion times. To address these challenges, Morbius is proposed, a scheduler that explicitly leverages the spatial structure of service applications to enhance scheduling decisions. Morbius adopts an all-or-nothing scheduling policy, ensuring that all tasks of an application are scheduled to run simultaneously, thereby effectively mitigating the non-synchronization problem. Within each priority queue, Morbius follows a shortest total time first policy, which facilitates contention-aware scheduling. Moreover, Morbius incorporates work conservation and starvation avoidance policies to better handle execution uncertainties and further improve application completion times. A prototype of Morbius is implemented on Yarn and evaluated in two environments: a homogeneous cluster with 36 machines and a heterogeneous cluster with 122 machines. Experimental results show that Morbius significantly outperforms existing approaches, improving average application completion time by up to 10.41x and reducing makespan by over 32.80%.

In this paper, we apply deep learning to the channel estimation problem of OFDM. Precisely speaking, to reduce the influence of noise on LS channel estimation, we design a channel estimation model based on attention mechanism and residual network. AttRNet-Conv and AttRNet-FC use attentional mechanism to change the weight of the feature to suppress noise. Compared with other channel estimation algorithms based on deep learning, AttRNet-Conv and AttRNet-FC avoid damaging channel information. The simulation results show that AttRNet-Conv and AttRNet-FC are superior to existing channel estimation algorithms based on deep learning in both high and low pilot frequency conditions. Even at low SNR, the estimation accuracy is higher than that of LMMSE. Experimental results show thatAttRNet-Conv and AttRNet-FC has strong robustness. In addition, we find that AttRNet-Conv is more suitable for channel estimation at low SNR in EPA, and the AttRNet-FC model is used in other cases.

Realistic fine-grained traffic traces are valuable to numerous applications in both academia and industry. However, obtaining them directly from devices is significantly challenging, while coarse-grained counters are readily available on almost all network devices. None of existing work can restore fine-grained traffic traces from counters, which we call network traffic super-resolution (TSR). To this end, we propose ZOOMSYNTH, the first TSR system that can achieve packet-level trace synthesis with counter traces as input. Following the basic structure of the TSR task, we design the Granular Traffic Transformer (GTT) model and the Composable Large Traffic Model (CLTM). CLTM is a tree of GTT models, and the GTT models in each layer perform upscaling on a particular granularity, which allows each GTT model to capture the traffic characteristics at this resolution. Using CLTM, we synthesize fine-grained traces from counters. We also leverage a rule-following model to comprehend counter rules (e.g. ACLs) when available, guiding the generations of fine-grained traces. We implement ZOOMSYNTH and perform extensive evaluations. Results show that, with only second-level counter traces, ZOOMSYNTH achieves synthesis quality comparable to existing solutions that takes packet-level traces as input. CLTM can also be fine-tuned to support downstream tasks. For example, ZOOMSYNTH with fine-tuned CLTM outperforms the existing solution by 27.5% and 9.8% in anomaly detection and service recognition tasks, respectively. To promote future research, we release the pre-trained CLTM-1.8B model weights along with its source code.

Large amounts of time-series data need to be continually delivered from IoT devices to the cloud for real-time data analytics. The data delivery process is intrinsically slow and costly. Therefore, lots of work proposes various data reduction methods to accelerate it. Yet, they are either designed for the simple linear time-series data or computation-intensive, which is not suitable for the IoT devices with limited resources. In this paper, we propose Xender, a system to accelerate time-series data delivery. Xender consists of two key components: data sampler and data generator. Data sampler works on IoT devices to sample time-series data with low resource footprint, and data generator works on the cloud to efficiently generate data that significantly resembles the original. Besides, Xender can adapt to the dynamic characteristics of the time-series data with the content-aware mechanism, as well as the dynamic computation resources by supporting multiple data generation quality levels and using the anytime generation mechanism. We implement Xender and evaluate it with testbed experiments using six real-world datasets. The results show that it can significantly reduce data delivery time by 45.79% on average compared against existing schemes, and adapt to computation resources with up to 1014.40Mbps data generation throughput.

Atmospheric and bulk deposition samples collected at Leigongshan Nature Reserve (LNR) were analyzed to explore the status, transport, and deposition of atmospheric pollution—specifically, organochlorine compounds—in the foggy mountains of southwestern China. Prohibiting and restricting persistent organic pollutants in the surrounding areas has led to a decline in organochlorine pesticides and polychlorinated biphenyls (PCBs), but not hexachlorobenzene (HCB), in the air. Wet deposition is a factor that strongly influences the atmospheric input at LNR. Despite the relatively low atmospheric concentration levels of hexachlorocyclohexanes (HCHs) and HCB, heavy precipitation can increase the deposition of these compounds in summer. The airborne HCHs exhibited a “bimodal” pattern, with higher concentrations in spring and autumn and lower concentrations in summer, the latter of which are results of high surface input into the ground along the transport routes of these compounds from South and Southeast Asia to southwestern China. The HCB at LNR mainly originates in mainland China. The deposition fluxes are influenced by the enhanced washout of atmospheric particles in the rainy season and increased particle-associated content in winter. The potential sources of o,pʹ-DDT, p,pʹ-DDT, and PCBs are distributed widely over low-latitude western areas and eastern China. Atmospheric transport and deposition in spring and summer contribute a significant proportion of the total fluxes annually when air masses originating in Southeast Asia arrive, increasing rainfall at LNR. The foggy local weather may also enhance this deposition, eventually leading to the accumulation of pollutants at LNR.

Federated learning has gained significant attention as a distributed machine learning paradigm, particularly due to its ability to preserve privacy by keeping data on IoT devices and only transmitting model parameters to the central server. However, existing federated learning systems face several challenges in their deployment. These challenges can be categorized into three main areas: data relation obliviousness, model characteristic obliviousness, and low communication bandwidth. To address these issues, we propose Marvel, a novel consolidated federated learning system designed specifically for efficient model training on IoT devices. Marvel tackles the challenges by employing a series of innovative techniques. Firstly, it clusters devices based on their data similarities, leveraging data fingerprints and the Hamming distance between them. This clustering process allows Marvel to group devices with similar data, enabling more efficient model training. Once the device clustering is complete, Marvel initiates the formal model training process. During this process, Marvel dynamically selects one device from each cluster to upload model parameters based on the predicted model convergence speed. This prediction is achieved using a lightweight LSTM-based prediction model, which helps determine the most suitable devices for parameter updates. Furthermore, Marvel employs a Tucker decomposition approach to decompose the parameters before transmission in each communication round. This decomposition reduces the size of the transmitted data and enables faster communication between devices and the central server. The parameters are then reconstructed on the central server, and vice versa when dispatching model parameters to devices. We evaluate Marvel using a 180-device testbed and implement a prototype using PyTorch. The results demonstrate the effectiveness of Marvel, as it achieves an average model training speedup of 59.81% and reduces communication rounds by up to 66.67% compared to existing schemes.

The Configuration Knowledge Base (CKB) facilitates network configuration management in multi-vendor environments by storing configuration snippet templates and pa-rameter settings for various vendors and intents, simplifying the deployment of the same configuration intent across different vendor devices. Current CKB construction methods follow a top-down manner, which requires handcrafted parameter labeling and snippet template creation for every new vendor or intent, resulting in significant expert effort. We seek to reduce this heavy human effort by leveraging our insights that different vendors share similar underlying intents and corresponding parameter types despite their different configuration languages, motivating bottom-up CKB construction manners which automatically create and align snippet templates of the same intent across all vendors by analyzing already existing device manuals and network configuration files. More specifically, we introduce ABC an active-learning-based tool that utilizes NLP models to produce snippet templates with manuals, extract labeled parameters in existing configuration files, and align those templates with common configuration models by encoding and comparing manuals of different vendors. We believe that creating such a tool would be a significant step towards achieving highly automated network configuration, which would also serve as a solid foundation for future endeavors such as intent-based network configuration, network configuration synthesis, and network device assimilation.

Gearbox has a compact structure, a stable transmission capability, and high transmission efficiency. Thus, it is widely applied and used as a critical transmission system in intelligent manufacturing systems, such as machine tools and robotics. The gearbox usually operates in harsh and non-stationary working environments, making the gear surface prone to wear. The progression of gear surface wear may lead to severe gear failures, such as gear tooth breakage and root crack, potentially damaging the whole gear transmission system. Therefore, it is essential to assess the gear surface wear progression and predict its remaining useful life (RUL) in order to ensure the reliable operation of the gear transmission system. To this end, this paper developed a novel gear wear prognostic scheme based on vibration analysis for gear health management. More specifically, a novel health indicator (HI) is first developed for gear wear monitoring in the proposed prognostic scheme. The novel HI, inferred from the cyclic correntropy and Wasserstein distance (WD), can accurately reflect the wear-induced cyclic correntropy spectra distribution change over time. Therefore, the novel HI can robustly evaluate the gear wear severity with high accuracy. With the developed HI, a network, namely the optimized gated recurrent unit (GRU), is applied for predicting the gear transmission system RUL during surface wear progression. As for the optimized GRU network, the genetic algorithm (GA) is applied to find the optimal hyperparameters adap-tively, which can significantly improve the practicality of the developed prognostic scheme. To conclude, the developed prognostic scheme can effectively reveal the gear wear propagation characteristics and predict the RUL accurately. A series of endurance tests are conducted to verify the effectiveness of the developed prognostic scheme for gear health management in surface wear progression.

The proliferation of edge devices has pushed computing from the cloud to the data sources, and video analytics is among the most promising applications of edge computing. Running video analytics is compute- and latency-sensitive, as video frames are analyzed by complex deep neural networks (DNNs) which put severe pressure on resource-constrained edge devices. To resolve the tension between inference latency and resource cost, we present Polly, a cross-camera inference system that enables co-located cameras with different but overlapping fields of views (FoVs) to share inference results between one another, thus eliminating the redundant inference work for objects in the same physical area. Polly’s design solves two basic challenges of cross-camera inference: how to identify overlapping FoVs automatically, and how to share inference results accurately across cameras. Evaluation on NVIDIA Jetson Nano with a real-world traffic surveillance dataset shows that Polly reduces the inference latency by up to 71.4% while achieving almost the same detection accuracy with state-of-the-art systems.

3D object detection plays a pivotal role in many applications, most notably autonomous driving and robotics. These applications are commonly deployed on edge devices to promptly interact with the environment, and often require near real-time response. With limited computation power, it is challenging to execute 3D detection on the edge using highly complex neural networks. Common approaches such as offloading to the cloud induce significant latency overheads due to the large amount of point cloud data during transmission. To resolve the tension between wimpy edge devices and compute-intensive inference workloads, we explore the possibility of empowering fast 2D detection to extrapolate 3D bounding boxes. To this end, we present Moby, a novel system that demonstrates the feasibility and potential of our approach. We design a transformation pipeline for Moby that generates 3D bounding boxes efficiently and accurately based on 2D detection results without running 3D detectors. Further, we devise a frame offloading scheduler that decides when to launch the 3D detector judiciously in the cloud to avoid the errors from accumulating. Extensive evaluations on NVIDIA Jetson TX2 with real-world autonomous driving datasets demonstrate that Moby offers up to 91.9% latency improvement with modest accuracy loss over state of the art.

Real-time video conferencing systems have recently become indispensable tools. However, existing commodity video conferencing systems often fail to deliver a satisfactory quality of experience (QoE) due to discrepancies between the actual network throughput and the average bitrate of encoded videos. In this paper, we present a system called Tyrus that aims to achieve efficient real-time video conferencing. Tyrus adapts its delivery of B-frames based on the real network bandwidth and the average bitrate of encoded videos, effectively addressing the QoE loss caused by mismatch issues. This system can be easily implemented on top of prevalent video conferencing systems that utilize standard video encoders. Unlike traditional approaches that treat all frame types equally, Tyrus enables adaptive delivery of B-frames, prioritizing the allocation of bandwidth to other crucial video frames. By doing so, it reduces end-to-end conferencing latency, especially when the network throughput experiences significant fluctuations. Additionally, Tyrus defines the playing deadline for B-frames by considering factors such as delivery time, buffering time, and the progress of preceding frames. It accurately classifies frame types according to video coding standards and estimates bandwidth allocation on a frame-by-frame basis. Moreover, Tyrus proactively handles potential errors in B-frame playing deadline estimation, minimizing their impact on video conferencing performance. Our evaluation results, obtained through real implementation on top of WebRTC, demonstrate the effectiveness of Tyrus. On average, it reduces per-frame latency by 19.03%, video stalls by 22.36%, and improves bandwidth utilization by at least 10.29%.

Coflow scheduling is critical to data-parallel applications in data centers. While schemes like Varys can achieve optimal performance, they require a priori information about coflows which is hard to obtain in practice. Existing non-clairvoyant solutions like Aalo generalize least attained service (LAS) scheduling discipline to address this issue. However, they fail to identify the bottleneck flows in a coflow and tend to allocate excessive bandwidth to the non-bottleneck flows, leading to bandwidth wastage and inferior overall performance. To this end, we present Fai that strives to improve the overall coflow performance by accelerating the bottleneck flows without priori knowledge. Fai employs bottleneck-aware scheduling. It adopts loose coordination to update coflow priority and flow rates based on total bytes sent. In addition, Fai detects bottleneck flows based on a flow’s rate and bytes sent, and de-allocates bandwidth for other flows to match the bottleneck rate without affecting the coflow completion time (CCT). The saved bandwidth is then distributed among coflows according to their priority to improve overall performance. Testbed evaluation on a 40-node cluster shows that Fai improves average (P95) CCT by 1.73× (3.43×), compared to Aalo. Large-scale trace-driven simulations also show that Fai outperforms Aalo substantially.

Molecular-level understanding of the exchange processes of heavy metals into the interlayer region of clay minerals help to predict the geochemical cycle of these elements. In this study, we conducted molecular dy-namics by using the newly developed CLAYFF-MOH parameters to elucidate the hydration characteristics of edge surfaces. On this basis, we explored the exchange processes of a series of heavy metal cations. Results showed that Mg for Al isomorphous substitution in the secondary outermost positions would reduce the coordination number (CN) of edge Al on both edge surfaces; water films were formed onto the interface owing to H-bonding interaction. Moreover, the cation exchange rates were Pb2+ > Cr3+ > Cd2+ > Ag+ > Ni2+ > Cu2+; Ag+ showed the largest exchange amount, followed by Ni2+, Cd2+, Pb2+, Cu2+, and Cr3+. A small part of Cl- was exchanged along with multivalent cations. The simulations of Ni2+-Cd2+-Pb2+ system indicated that coexistence of divalent cations had no significant effect on the exchange, while the exchange process in Ag+-Cd2+-Cr3+ mixing system was more complicated. Electrostatic interaction was the dominant driving force responsible for exchange. Additionally, Na+ and Ag+ were complexed on the octahedral vacancy, Si-OH, and Al-OH sites. The mobilities of cations decreased with increased ionic valence increased, and overall the diffusion in the interlayer was more constrained than in external solution. This study provides fundamental molecular-level knowledge of the fate of heavy metals in natural environment.

On-boarding new devices into an existing SDN network is a pain for network operations (NetOps) teams, because much expert effort is required to bridge the gap between the configuration models of the new devices and the unified data model in the SDN controller. In this work, we present an assistant framework NAssim, to help NetOps accelerate the process of assimilating a new device into a SDN network. Our solution features a unified parser framework to parse diverse device user manuals into preliminary configuration models, a rigorous validator that confirm the correctness of the models via formal syntax analysis, model hierarchy validation and empirical data validation, and a deep-learning-based mapping algorithm that uses state-of-the-art neural language processing techniques to produce human-comprehensible recommended mapping between the validated configuration model and the one in the SDN controller. In all, NAssim liberates the NetOps from most tedious tasks by learning directly from devices' manuals to produce data models which are comprehensible by both the SDN controller and human experts. Our evaluation shows, NAssim can accelerate the assimilation process by 9.1x. In this process, we also identify and correct 243 errors in four mainstream vendors' device manuals, and release a validated and expert-curated dataset of parsed manual corpus for future research.

Nowadays, various network telemetry technologies are proposed to monitor the network and detect failures accurately in real-time, which can be categorized into two types, including the proactive network telemetry (NT) and the passive one. The passive NT can monitor the network with low band-width overhead, yet, cannot guarantee full network coverage. The proactive one can achieve full coverage, yet, lead to high bandwidth cost. To deal with the problem, we propose a failure-aware clustered network telemetry approach, called CNT. CNT leverages the practical objective network operating experience: different network links have various failure probabilities. It is aware of the failure probabilities and assigns the network links into two clusters accordingly. Then, based on the original network topology, CNT designs an active path planning algorithm to connect the two clusters of links into two sub-topologies, respectively. Finally, CNT performs network telemetry with different cycles. We evaluate CNT with various simulation experiments. The results show that compared to existing proactive schemes, CNT can achieve comparable network coverage with less cost.

Network function virtualization (NFV) enables elastic scaling to middlebox deployment and management. Therefore, efficient stateful scaling is an important task because operators often need to shift traffic and the associated flow states across VNF instances to deal with time-varying loads. Existing NFV scaling methods, however, typically focus on one aspect of the scaling pipeline and does not offer an end-to-end scaling framework. This article presents ScaleFlux, a complete stateful scaling system that efficiently reduces flow-level latency and achieves near-optimal resource usage. ScaleFlux (1) monitors traffic load for each VNF instance and adopts a queue-based mechanism to detect load burstiness timely, (2) deploys a flow bandwidth predictor to predict flow bandwidth time-series with the ABCNN-LSTM model, and (3) schedules the necessary flow and state migration using the simulated annealing algorithm to achieve both flow-level latency guarantee and resource usage minimization. Testbed evaluation with a five-machine cluster shows that ScaleFlux reduces flow completion time by at least 8.7× for all the workloads and achieves near-optimal CPU usage during scaling.