The advent of architectures with software-exposed resources (Spatial Architectures) has created a demand for universally applicable scheduling techniques. This paper describes our generalized spatial scheduling framework, formulated with Integer Linear Programming, and specifically accomplishes two goals. First, using the "Simple" architecture, it illustrates how to use our open-source tool to create a customized scheduler and covers problem formulation with ILP and GAMS. Second, it summarizes results on the application to three real architectures (TRIPS,DySER,PLUG), demonstrating the technique's practicality and competitiveness with existing schedulers.

Data structure lookups are among the most expensive operations on routers' critical path in terms of latency and power. Therefore, efficient lookup engines are crucial. Several approaches have been proposed, based on either custom ASICs, general-purpose processors, or specialized engines. ASICs enable high performance but have long design cycle and scarce flexibility, while general-purpose processors present the opposite trade-off. Specialized programmable engines achieve some of the benefits of both approaches, but are still hard to program and limited either in terms of flexibility or performance. In this paper we investigate a different design point. Our solution, SWSL (SoftWare Synthesis for network Lookup) generates hardware logic directly from lookup applications written in C++. Therefore, it retains a simple programming model yet leads to significant performance and power gains. Moreover, compiled application can be deployed on either FPGA or ASIC, enabling a further trade-off between flexibility and performance. While most high-level synthesis compilers focus on loop acceleration, SWSL generates entire lookup chains performing aggressive pipelining to achieve high throughput. Initial results are promising: compared with a previously proposed solution, SWSL gives 2-4 × lower latency and 3-4× reduced chip area with reasonable power consumption.

Specialized execution using spatial architectures provides energy efficient computation, but requires effective algorithms for spatially scheduling the computation. Generally, this has been solved with architecture-specific heuristics, an approach which suffers from poor compiler/architect productivity, lack of insight on optimality, and inhibits migration of techniques between architectures. Our goal is to develop a scheduling framework usable for all spatial architectures. To this end, we expresses spatial scheduling as a constraint satisfaction problem using Integer Linear Programming (ILP). We observe that architecture primitives and scheduler responsibilities can be related through five abstractions: placement of computation, routing of data, managing event timing, managing resource utilization, and forming the optimization objectives. We encode these responsibilities as 20 general ILP constraints, which are used to create schedulers for the disparate TRIPS, DySER, and PLUG architectures. Our results show that a general declarative approach using ILP is implementable, practical, and typically matches or outperforms specialized schedulers.

Table lookups and other types of packet processing require so much memory bandwidth that the networking industry has long been a major consumer of specialized memories like TCAMs. Extensive research in algorithms for longest prefix matching and packet classification has laid the foundation for lookup engines relying on area- and power-efficient random access memories. Motivated by costs and semiconductor technology trends, designs from industry and academia implement multi-algorithm lookup pipelines by synthesizing multiple functions into hardware, or by adding programmability. In existing proposals, programmability comes with significant overhead. We build on recent innovations in computer architecture that demonstrate the efficiency and flexibility of dynamically synthesized accelerators. In this paper we propose LEAP, a latency-energy- and area- optimized lookup pipeline based on an analysis of various lookup algorithms. We compare to PLUG, which relies on von-Neumann-style programmable processing. We show that LEAP has equivalent flexibility by porting all lookup algorithms previously shown to work with PLUG. At the same time, LEAP reduces chip area by 1.5×, power consumption by 1.3×, and latency typically by 5×. Furthermore, programming LEAP is straight-forward; we demonstrate an intuitive Python-based API.

We present a software-based solution to the multi-dimensional packet classification problem which can operate at high line speeds, e.g., in excess of 10 Gbps, using high-end multi-core desktop platforms available today. Our solution, called Storm, leverages a common notion that a subset of rules are likely to be popular over short durations of time. By identifying a suitable set of popular rules one can significantly speed up existing software-based classification algorithms. A key aspect of our design is in partitioning processor resources into various relevant tasks, such as continuously computing the popular rules based on a sampled subset of traffic, fast classification for traffic that matches popular rules, dealing with packets that do not match the most popular rules, and traffic sampling. Our results show that by using a single 8-core Xeon processor desktop platform, it is possible to sustain classification rates of more than 15 Gbps for representative rule sets of size in excess of 5-dimensional 9000 rules, with no packet losses. This performance is significantly superior to a 8-way implementation of a state-of-the-art packet classification software system running on the same 8-core machine. Therefore, we believe that our design of packet classification functions can be a useful classification building block for RouteBricks-style designs, where a core router might be constructed as a mesh of regular desktop machines.

A Network Intrusion Detection System (NIDS) resides at the edge of a network and is tasked with identifying and removing all occurrences of malicious traffic traversing the network. At its heart is a signature matching engine that compares each byte of incoming and outgoing traffic to signatures representing known vulnerabilities or exploits and flags or drops traffic that matches a signature. Signature Matching is fundamentally a language recognition problem. Signatures are commonly represented as regular expressions, which can be matched simultaneously by finite automata. Unfortunately, standard nondeterministic finite automata (NFAs) and deterministic finite automata (DFAs) induce a space-time tradeoff when combined and are unsuitable for NIDS use. NFAs are small and compact but too slow, whereas DFAs are fast but consume too much memory. Other alternatives such as filtering induce a tradeoff between accuracy and execution time. We argue that the tradeoffs associated with signature matching are not fundamental obstacles, but instead result from limitations in existing models. We posit that with richer, more complex matching models, we can devise mechanisms that obviate these tradeoffs and have acceptable memory and performance profiles. In an analysis of an existing NIDS, we show that the use of filters induces worst-case behavior that is six orders of magnitude slower than the average case and can be invoked by an adversary to enable perpetual evasion. We present an enhanced model, with an acceptably small increase in memory, that brings the worst case to within one order of magnitude of the average case. Next, for DFA-based matching we present a first-principles characterization of the state-space explosion and subsequent memory exhaustion that occurs when DFAs are combined, and we give conditions that eliminate it when satisfied. We then show that through the careful inclusion of auxiliary state variables, we can transform automata so that they satisfy these conditions and do not lead to memory exhaustion. With this enriched model, we achieve matching speeds approaching DFAs with memory footprints similar to NFAs. A NIDS operates in a constrained, adversarial environment. This work makes contributions towards enabling robust signature matching in such environments.

Applications sensitive to service quality, such as interac tive video, could fuel the growth of the Internet if the network could support them. ISPs do provide QoS within VPNs, but endhosts connected to the public Internet and linked by paths that cross multiple ISPs can do little to influence the quality of the service for their traffic. While over-provisioning helped eliminate congestion in backbones, congestion is still present at the edges and it is not clear that if inter-domain QoS were feasible, major overprovisioning would remain the most cost-effective solution. RFC 3869 “IAB Concerns and Recommendations Regarding Internet Research and Evolution” [AF04] identifies two main open problems that must be solved by a future inter-domain QoS architecture. “Deploying existing QoS mechanisms (...) across an inter-domain boundary creates a significant and easily exploited denial-of-servi c vulnerability for any network that provides inter-domain QoS support. (...) Also, current business models are not consistent with inter-domain QoS, in large part because it is impractical or impossible to authenticate the identit y of the sender of would-be preferred traffic while still forwarding traffic at line-rate. Absent such an ability, it is unclear how a network operator could bill or otherwise recover costs associated with providing that preferred ser vice.” A distributed accounting system owned and operated by ISPs could solve the billing problem. We list below in rough order of importance the properties we believe such a system should have.

Over the past couple of decades, the Internet has rapidly evolved from a collaborative social experiment to an agglomerate of competing commercial providers. This shift has helped maintain growth and has turned the Internet into a vast economic force, but it has also introduced some serious problems. A particularly bad problem is the inability of end-users to obtain the desired levels of performance for their transfers. Today, an organization can set up a contract with its ISP to ensure that the ISP offers good service to its traffic. But typical transfers in the Internet traverse multiple ISPs and it is clearly infeasible for the organization to have contracts with all of them. It is possible for neighboring ISPs to enter into contracts that require them to offer good performance to each other’s “premium” traffic. However, such contracts are extremely rare and, even when used, cannot guarantee good end-to-end service to user transfers. Our thesis in this paper is that we can support good end-to-end service to user transfers by extending the current model of binding bi-lateral contracts between neighboring entities (e.g. customers and providers or peering partners) with simple mechanisms that produce tacit incentives for remote ISPs. Our use of the phrase “good end-to-end service” is intentional: our goal is not to offer “end-to-end QoS” with strict performance guarantees, but rather to provide end users the flexibility to improve the performance experienced by their transfers, as and when desired. Our proposal builds on two main end-user based mechanisms that generate the tacit incentives.

Good performance under extreme workloads and isolation between the resource consumption of concurrent jobs are perennial design goals of computer systems ranging from multitasking servers to network routers. In this paper we present a specialized system that computes multiple summaries of IP traffic in real time and achieves robustness and isolation between tasks in a novel way: by automatically adapting the parameters of the summarization algorithms. In traditional systems, anomalous network behavior such as denial of service attacks or worms can overwhelm the memory or CPU, making the system produce meaningless results exactly when measurement is needed most. In contrast, our measurement system reacts by gracefully degrading the accuracy of the affected summaries.The types of summaries we compute are widely used by network administrators monitoring the workloads of their networks: the ports sending the most traffic, the IP addresses sending or receiving the most traffic or opening the most connections, etc. We evaluate and compare many existing algorithmic solutions for computing these summaries, as well as two new solutions we propose here: "flow sample and hold" and "Bloom filter tuple set counting". Compared to previous solutions, these new solutions offer better memory versus accuracy tradeoffs and have more predictable resource consumption. Finally, we evaluate the actual implementation of a complete system that combines the best of these algorithms.

Network worms are a major threat to the security of today’s Internet-connected hosts and networks. The combination of unmitigated connectivity and widespread software homogeneity allows worms to exploit tremendous parallelism in propagation. Modern worms spread so quickly that no human-mediated reaction to the outbreak of a new worm can hope to prevent a widespread epidemic. In this paper we propose an automated method for detecting new worms based on traffic characteristics common to most of them: highly repetitive packet content, an increasing population of sources generating infections and an increasing number of destinations being targeted. Our method generates content signatures for the worm without any human intervention. Preliminary results on a small network show promising results: we have identified three confirmed worms with a low percentage of false positives. This gives us reason to believe that our method could form the core of an effective network-level worm detection and countermeasure system capable of substantially slowing down the spread of new worms.

Monitoring traffic on important links allows network administrators to get insights into how their networks are used or misused. Traffic analysis based on NetFlow records or packet header traces can reveal floods, aggressive worms, large (unauthorized) servers, spam relays, and many other phenomena of interest. Existing tools can plot time series of pre-defined traffic aggregates, or perform (hierarchical) "heavy hitter" analysis of the traffic. Wisconsin Netpy is a software package that goes beyond the capabilities of other existing tools through its support for interactive analysis and novel powerful visualization of the traffic data. Adaptive sampling of flow records ensures that the performance is good enough for interactive use, while the results of the analyses stay close to the results based on exact data. Among the salient features of the package are: hierarchical analyses of source addresses, destination addresses, or applications within aggregates identified by user-defined filters; time series plots that separate the traffic into categories specified with ACL-like syntax at run time; interactive drill-down into analyses of components of the traffic mix; "heatmap" visualization of traffic that describes how two "dimensions" of the traffic relate to each other (e.g., which sources send to which destinations, or which sources use which service, etc.).

Internet links operate at high speeds, and past trends predict that these speeds will continue to increase rapidly. Routers and Intrusion Detection Systems that operate at up to OC-768 speeds (40 Gigabits/second) are currently being developed. In this paper we address a basic function common to several security and measurement applications running at line speeds: counting the number of distinct header patterns (flows) seen on a high speed link in a specified time period. For example one can detect port scans by counting the number of connections opened by the suspected port scanner and one can ascertain that a denial of service attack is in progress by counting the number of distinct (fake) source addresses that send packets to the suspected victim. We provide algorithms solving the flow counting problem using extremely small amounts of memory. This can translate into savings of scarce fast memory (SRAM) for hardware implementations. It can also help systems that use cheaper DRAM to allow them to scale to larger instances of the problem. The reduction in memory is particularly important for network security applications such as detecting port scans and DoS attacks that need to run a separate instance of the algorithm for each suspected attacker or victim. Our algorithms can be implemented in hardware at wire speeds (8 nsec per packet for OC-768) using simple CRC based hash functions, multiplexers and SRAM. They access at most one or two memory locations per packet. We expose the behavior of simple building blocks and provide a family of customizable counting algorithms that can be adapted to the requirements of various applications. This allows our algorithms to use less memory than the best known counting algorithm, probabilistic counting, to provide the same accuracy.• Virtual bitmap is well suited for triggers (which need only be accurate around a threshold value) such as detecting DoS attacks, and uses 292 bytes to achieve an error of 2.619% compared to 5,200 bytes for probabilistic counting.• Our "no assumptions" counting algorithm, multiresolution bitmap algorithms uses only 2143 bytes to count up to 100 million flows with an average error of 3%. Its accuracy is slightly better when the number of flows is small, while the accuracy of probabilistic counting using the same memory is much worse.• Our adaptive bitmap that exploits stationarity in the number of flows can count the number of distinct flows on a link that contains anywhere from 0 to 100 million flows with an average error of less than 1 % using only 2 Kbytes of memory. Probabilistic counting needs eight times more memory to achieve the same accuracy.• We used our triggered bitmap to replace the port scan detection component of the popular intrusion detection system Snort. Doing so reduced the memory usage from 89 Mbytes to 6.4 Mbytes (with an average error of 13.5%) as measured on a 10 minute trace. Probabilistic counting uses 23 Mbytes.This work was made possible by a grant from NIST for the Sensilla Project.

Accurate network traffic measurement is requir ed for accounting,bandwidth provisioning, and detecting DOS attacks. However, keeping a counter to measure the traffic sent by eachof a million concurrent flows is too expensi ve (using SRAM) or slow (using DRAM). The current state-of-the-art (e.g., Cisco NetFlow) methods which count periodically sampledpacketsareslow, inaccurate,and memory-intensive. Our paper intr oducesa paradigm shift by concentratingon the problemof measuringonly ”heavy” flows— i.e.,flowswhosetraffic is abovesomethr esholdsuch as 1% of the link. After showing that a number of simple solutions basedon cachedcounters and classicalsampling do not work, we describe two novel and scalableschemes for this purposewhich take a constant number of memory referencesper packet and usea small amount of memory. Further, unlik eNetFlow estimates,wehaveprovablebounds on the accuracy of measured rates and the probability of false negatives. We also proposea new form of accounting called threshold accounting in which only flowsabovethr eshold are chargedby usagewhile the restare chargeda fixed fee. Thr esholdaccounting generalizesthe familiar notions of usage-basedand duration basedpricing.