The emergence of programmable switches such as the Intel Tofino has made it possible, in theory, to implement many network monitoring applications directly in the network data plane. In practice, however, such implementations are often more challenging than expected. A key difficulty is that such applications often depend, in part, on recognizing traffic patterns that are easy to specify as a deterministic finite state automaton (a DFA) but hard to implement thanks to stringent hardware constraints: to maximize throughput and avoid race conditions, state machine updates must be completed in a single Tofino pipeline stage, but the limited computational resources make finding an implementation a challenging puzzle. This paper presents a solution to such puzzles---a general framework for synthesizing DFA implementations automatically. A key insight is that such a synthesis system is free to renumber state machine states and implement transitions using any available arithmetic or logical operations over that renumbering, provided the resulting implementation is semantically equivalent to the input specification. To produce such a synthesizer, we model the required state machine semantics and the available single-stage switch operations using SMT constraints. An off-the-shelf SMT solver finds a solution to the constraints, and this solution is then translated to P4 code. We evaluate the effectiveness of our methods by synthesizing state machines for a variety of useful applications, including those that monitor TCP handshakes and video conference streams.
We conduct a systematic review of two decades of work on wide-area network traffic engineering (TE). We summarize the contributions of important TE algorithms and systems developed for Internet Service Provider and cloud wide-area networks. We study the evolution of the goals of TE (from performance to reliability), TE system design (from decentralized to fully-centralized to partly-centralized) and the technology used in deploying these systems in large commercial networks (from vendor-specific protocols to software-defined implementations). We define a taxonomy of TE systems to categorize developments in TE research. We identify trends at the forefront of TE research and practice to motivate an agenda for future work in this area. Finally, to aid future work, we are releasing our summaries and implementations of several recent TE algorithms. 1 .
Packet drops caused by congestion are a fundamental problem in network operation. Yet, it is difficult to detect where drops are happening, let alone which flows are most affected. Detecting the small-timescale drops caused by short bursts of traffic is even more challenging, and traditional monitoring techniques can easily miss them. To uncover packet drops as they occur inside a switch, the analysis must be real-time, fine-grained, and efficient. However, modern switches have distributed packet-processing pipelines that see either the arriving or departing traffic, but not the packet drops. Additionally, they do not have enough memory to store per-flow state. Our MIDST system addresses these challenges through a distributed compact data structure with lightweight coordination between ingress and egress pipelines. MIDST identifies the flows experiencing loss, as well as the bursty flows responsible, across different burst durations. Our evaluation with real-world traces and TCP connections shows that MIDST uses little memory (e.g., 320KB) while providing high accuracy (95% to 98%) under varying loss rates and burst durations. We evaluate a low-rate DDoS attack and demonstrate the potential use of our measurement results for attack detection and mitigation.
Distributed discrete-event simulation is an essential method for analyzing large-scale models, including weather forecast and network simulations. A distributed simulation often requires synchronizing state among the different parts of the model according to a global virtual time (GVT). However, existing approaches require multiple round-trip times to a server to compute a new GVT value. In this paper, we propose NetGVT, a system that computes GVT using programmable switches, thereby avoiding the round-trip latency of a server-based solution. In particular, our design is concerned with two main constraints of the switch programming model: the limited number of arithmetic and logic operations and the limited memory available on the device. We aggregate computations and unroll them across different pipeline stages in a hierarchical manner to address the former. Then, we adopt compression mechanisms to store a short representation of virtual clocks in the on-chip registers to tackle the memory limitations. We implemented a prototype of NetGVT and evaluated its performance with a synthetic lock-step simulation in a Tofino switch. Our results demonstrate that NetGVT outperforms techniques that do not rely on in-network computing by 40% in terms of distributed simulations completion time.
The latest generation of 5G telecommunication networks are expected to provide high throughput and low latency while catering to diverse applications like mobile broadband, dense IoT, and self-driving cars. A high performance User Plane Function (UPF), the main element in the 5G user plane, is critical to achieving these performance goals. This paper presents AccelUPF, a 5G UPF that offloads functionality to programmable dataplane hardware for performance acceleration. While prior work has proposed accelerating the UPF by offloading its data forwarding functionality to programmable hardware, the Packet Forwarding Control Protocol (PFCP) messages from the control plane that configure the hardware data forwarding rules were still processed in software. We show that only offloading data forwarding and not PFCP message processing leads to suboptimal performance in the UPF for applications like IoT that have a much higher ratio of PFCP messages to data traffic, due to a bottleneck at the software control plane that configures the hardware packet forwarding rules. In contrast to prior work, AccelUPF offloads both PFCP message processing as well as data forwarding to programmable hardware. AccelUPF overcomes several technical challenges pertaining to the processing of the complex variable-sized PFCP messages within the memory and compute constraints of programmable hardware platforms. Our evaluation of AccelUPF implemented over a Netronome programmable NIC and an Intel Tofino programmable switch demonstrates performance gains over the state-of-the-art UPFs for real-world traffic scenarios.
Over the last decade, programmable data planes have enabled highly customizable and efficient packet processing in commercial off-the-shelf hardware. Although researchers have demonstrated various use cases of this technology, its potential misuse has gained much less traction. This work investigates a typical surveillance scenario, VoIP call identification and monitoring, through a tailored data-plane attack. We introduce DELTA, a network-level side-channel attack that can efficiently identify VoIP calls and their hosting services. DELTA achieves this by tracking the inherent network footprint of VoIP services in the data plane. Specifically, DELTA stores the user addresses recently connected to VoIP services and links potential call flows with these addresses. We implement DELTA on existing hardware and conduct high-throughput tests based on representative traffic. DELTA can simultaneously store around 100 000 VoIP connections per service and identify call streams in-path, at line-rate, inside terabits of Internet traffic per second, immediately revealing users' communication patterns.
Designing networked systems that take best advantage of heterogeneous dataplanes - e.g. , dividing packet processing across both a PISA switch and an x86 CPU - can improve performance, efficiency, and resource consumption. However, programming for multiple hardware targets remains challenging because developers must learn platform-specific languages and skills. While some 'write-once, run-anywhere' compilers exist, they are unable to consider a range of implementation options to tune the NF to meet performance objectives. In this short paper, we explore preliminary ideas towards a compiler that explores a large search space of different mappings of functionality to hardware. This exploration can be tuned for a programmer-specified objective, such as minimizing memory consumption or maximizing network throughput. Our initial prototype, SyNAPSE, is based on a methodology called component-based synthesis and supports deployments across x86 and Tofino platforms. Relative to a baseline compiler which only generates one deployment decision, SyNAPSE uncovers thousands of deployment options - including a deployment which reduces the amount of controller traffic by an order of magnitude, and another deployment which halves memory usage.
The Domain Name System (DNS) is a significant component of modern-day internet. Despite this fact, DNS traffic is mostly unencrypted, and as such a likely target for exploitation by malicious actors. The advancement of programmable switches presents researchers with the opportunity to explore DNS traffic from a new vantage point, without sacrificing network bandwidth. In spite of the incentive, DNS research in programmable switches has been scarce, owing to the difficulty in parsing DNS packets. We present a general solution to DNS packet parsing that can handle the vast majority of DNS packets (97%) using current hardware and can easily be scaled to parse all DNS packets as hardware improves. Our highly configurable solution can be adjusted to fit many distinct use cases. Additionally, we explore the challenges involved in parsing DNS packets and present common pitfalls appearing in previous research attempting to do so.
Emerging software-defined networking technologies enable more adaptive communication infrastructures, allowing for quick reactions to changes in networking requirements by exploiting the workload's temporal structure. However, operating networks adaptively is algorithmically challenging, as meeting networks' stringent dependability requirements relies on maintaining basic consistency and performance properties, such as loop freedom and congestion minimization, even during the update process. This paper leverages an augmentation-speed tradeoff to significantly speed up consistent network updates. We show that allowing for a small and short (hence practically tolerable, e.g., using buffering) oversubscription of links allows us to solve many network update instances much faster, as well as to reduce computational complexities (i.e., the running times of the algorithms). We first explore this tradeoff formally, revealing the computational complexity of scheduling updates. We then present and analyze algorithms that maintain logical and performance properties during the update. Using an extensive simulation study, we find that the tradeoff is even more favorable in practice than our analytical bounds suggest. In particular, we find that by allowing just 10% augmentation, update times reduce by more than 32% on average, across a spectrum of real-world networks.
Guaranteeing high availability of networks virtually hinges on the ability to handle and recover from bugs and failures. Yet, despite the advances in verification, testing, and debugging, production networks remain susceptible to large-scale failures --- often due to deterministic bugs. This paper explores the use of input transformations as a viable method for recovering from such deterministic bugs. In particular, we introduce an online system, Tardis, for overcoming deterministic faults by using a blend of program analysis and runtime program data to systematically determine the fault-triggering input events and using domain-specific models to automatically generate transformations of the fault-triggering inputs that are both safe and semantically equivalent. We evaluated Tardison several production network control plane applications (CPAs), including six SDN CPAs and several popular BGP CPAs using 71 realistic bugs. We observe that Tardisimproves recovery time by 7.44%, introduces a 25% CPU and 0.5% memory overhead, and recovers from 77.26% of the injected realistic and representative bugs, more than twice that of existing solutions.
Sketching algorithms or sketches are attractive as telemetry capabilities on programmable hardware switches since they offer rigorous accuracy guarantees and use compact data structures. However, we find that in practice, their actual implementations can have a significant (up to 94×) accuracy drop compared to theoretical expectations. We find that the delays incurred by pulling and resetting the data plane state induce accuracy degradation. We design and implement solutions to reduce the delays and show that our solutions can help eliminate almost all the inaccuracy of existing sketch workflows.
The demands on mobile networks are constantly evolving, but designing and integrating new high-speed packet processing remains a challenge due to the complexity of requirements and opacity of protocol specifications. 5G data planes should be implemented in programmable hardware for both speed and flexibility, and extending or replacing these data planes should be painless. In this paper we implement the 5G data plane using two P4 programs: one that acts as a open-source model data plane to simplify the interface with the control plane, and one to run efficiently on hardware switches to minimize latency and maximize bandwidth. The model data plane enables testing changes made to the control plane before integrating with a performant data plane, and vice versa. The hardware data plane implements the fast path for device traffic, and makes use of microservices to implement functions that highspeed switch hardware cannot do. Our data plane implementation is currently in limited deployment on three university campuses where it is enabling new research on mobile networks.
In this paper, we introduce P4 Weaver as an approach towards bringing modularity into the P4 language. P4 Weaver is designed to merge new data plane features into a base program in a principled and controlled way, so as to preserve the reliability of the switch. We also present an architecture for an integrated development environment that supports modular P4 programming while also safeguarding the intellectual property of the vendor code. We demonstrate the utility of P4 Weaver by adding three popular but non-trivial protocols to a P4 switch. We show that modularity is indeed beneficial and that P4 Weaver supports modularity efficiently and reliably.
Clusteringis a basic machine learning task. In this task, a stream of input items needs to be grouped into clusters, such that all items classified into the same cluster are closer to each other than to items classified to other clusters. Each cluster is centered around a centroidpoint, which may either be given as a parameter, or must be learned during the process in the case of unsupervised online learning. This work studies the ability to perform clustering, e.g., for classifying network traffic, in programmable switches. Conducting such classification by the switches through which the traffic flows is potentially the most efficient approach. To that end, we develop Clustreams, a novel in-network clustering system designed to handle clustering in the data path. At the core of Clustreamsis a novel clustering algorithm that relies heavily on TCAM (Ternary Content Addressable Memory) match-action capabilities. This algorithm is realized for the Nvidia Spectrum-3 switch, and is limited to classification when the centroid points are known a-priori. The work includes accuracy measurements for the algorithms, as well as run-time performance measurements and analysis of the clustering algorithm on a Spectrum-3 switch. As shown in the measurements, Clustreamsobtains very high accuracy without any noticeable run-time impact on the switch' performance.
There is an emerging need for scalable high-performance in-networkrate-limiting because rate-limiters can be used to provide performance isolation. However, existing approaches to in-network rate-limiting are not scalable or TCP-friendly. This paper presents the design of Nimble, a new approach to in-network rate-limiting that is scalable, high performance, and TCP-friendly. Nimble uses meters to scalably provide hardware rate-limiting without any dedicated queuing or buffering resources, and Nimble uses ECN-Shaping for TCP-friendly rate-limit enforcement. Nimble also introduces the first algorithm for configuring in-network rate-limiters to enforce network-wide isolation policies. Through a P4 implementation and experiments with a 100Gbps Barefoot Tofino switch, we find that Nimble is immediately usable and can operate even with high bandwidth rate-limits without needing to recirculate packets or rely on hardware packet generators to generate token refill packets. This overcomes the scalability limitations of prior approaches. Experiments with Apache and Redis show that Nimble can reduce application-level latency by an order of magnitude when compared to not using in-network rate-limiting, and ns-3 simulations demonstrate that Nimble behaves well in larger clusters. We find that Nimble can scale to 100K rate-limiters perswitch when implemented on a Barefoot Tofino switch, and our new rate allocation algorithm reduces rate-limiter updates by a factor of 10x-24x and improves network utilization by 24%.
In networks today, the data plane handles forwarding--- sending a packet to the next device in the path---and the control plane handles routing---deciding the path of the packet in the network. This architecture has limitations. First, when link failures occur, the data plane has to wait for the control plane to install new routes, and packet losses can occur due to delayed routing convergence or central controller latencies. Second, policy-compliance is not guaranteed without sophisticated configuration synthesis or controller intervention. Fast reroute mechanisms in the data plane cannot provide both connectivity and policy-compliance guarantees. We take advantage of the recent advances in fast programmable switches to perform policy-compliant route computations entirely in the data plane, thus providing fast and programmable reactions to failures. D2R provides the illusion of a hierarchical network fabric that is always available and policy-compliant under failures. We implement our data plane in P4 and show its viability in real world topologies.
As we increasingly depend on networked services, ensuring resiliency of networks against network failures and providing bounded latency to applications become imperative. Adding ample redundancy in the network substrate alone is not sufficient; resilient routing mechanisms that can effectively take advantage of such topological diversity also play a critical role. In this paper, we present Taproot, a resilient diversity routing algorithmthat ensures bounded latencyfor packet delivery under failures by leveraging a preordeR@routing structure with precomputed routing rules. Leveraging the centralizedcontrol plane and programmable match-actionrules in the data plane, we describe how Taproot can be realized in SDN networks. We implement Taproot in OVS and conduct extensive simulations and experiments to demonstrate its superior performance over existing solutions. Our results show that by tuning the latency allowance upon failure, Taproot reduces/eliminates the number of disconnected src-dst pairs even under 10 link failures. Finally, as a use case, we illustrate the impact of control channel failures on SDN data plane/application performance, and employ Taproot to provide a "hardened" SDN control network with bounded latency against failures. Our results show that Taproot immediately detects the failure and re-routes the control messages to a different path avoiding failed links/nodes. Hence, the control channel is maintained without interruption, or involvement from the controller, and the throughput was not affected.
Transport protocols can be implemented in NIC (Network Interface Card) hardware to increase throughput, reduce latency and free up CPU cycles. If the ideal transport protocol were known, the optimal implementation would be simple: bake it into fixed-function hardware. But transport layer protocols are still evolving, with innovative new algorithms proposed every year. A recent study proposed Tonic, a Verilog-programmable transport layer in hardware. We build on this work to propose a new programmable hardware transport layer architecture, called nanoTransport, optimized for the extremely low-latency message-based RPCs (Remote Procedure Calls) that dominate large, modern distributed data center applications. NanoTransport is programmed using the P4 language, making it easy to modify existing (or create entirely new) transport protocols in hardware. We identify common events and primitive operations, allowing for a streamlined, modular, programmable pipeline, including packetization, reassembly, timeouts and packet generation, all to be expressed by the programmer. We evaluate our nanoTransport prototype by programming it to run the reliable message-based transport protocols NDP and Homa, as well as a hybrid variant. Our FPGA prototype - implemented in Chisel and running on the Firesim simulator - exposes P4-programmable pipelines and is designed to run in an ASIC at 200Gb/s with each packet processed end-to-end in less than 10ns (including message reassembly).
Associating network traffic with human-readable domain names, instead of low-level identifiers like IP addresses, is helpful for measuring traffic by domain name, rate-limiting packets by domain, and identifying IoT devices. However, existing monitoring techniques require examining traffic at an external compute node, introducing overhead and privacy risks. In this paper, we introduce Meta4, a framework for monitoring traffic by domain name in the data plane by extracting the client IP, server IP, and domain name from DNS response messages and associating the domain name with data traffic from the subsequent client-server session. A data-plane implementation has the benefits of running efficiently at line-rate, enabling the switch to take direct action on the packets (e.g., to rate-limit, block, or mark traffic based on the associated domain), and protecting the privacy of user information. We implemented Meta4 on an Intel Tofino switch and evaluated our prototype against packet traces from an operational network.
Deploying traffic engineering (TE) in the context of multi-controller SDN (MCSDN) or on WANs is challenging due to state and consistency requirements. For example, using strong consistency to ensure that information is always up-to-date introduces significant performance overheads. However, using eventual consistency to reduce synchronisation time comes at the expense of using outdated information to make decisions. We design and implement Helix, an MCSDN system that supports deployment on WANs. Helix offloads operations closer to the data plane and minimises shared state between devices, allowing it to tolerate high latency and mitigate state consistency concerns. We develop a lightweight TE algorithm that requires minimal state, making it suitable for use with Helix. Our simulation results show that Helix reduces congestion loss by up to 1.6x and performs 12x fewer path changes compared to CSPF.
