Artifact evaluation places a significant burden on both authors and reviewers: Authors must package bespoke experimental setups, while committees struggle to validate them. Research Infrastructures (RIs) offer tools to support reproducibility, yet interoperability remains limited due to insufficient standardization efforts. To address this, we propose new standards that unify RIs through a formal experiment execution format and topology management for networked experiments. In addition, we advance sustainability through integrated energy analysis and modeling. Together, these contributions improve reproducibility, enable cross-RI collaboration, and foster more sustainable experiments. This makes RIs more FAIR-particularly by strengthening interoperability and reusability-while aiding broader adoption across the (network) research community.
Reproducing experimental results in computing and networking is notoriously difficult due to intricate dependencies and diverse execution environments. Re-executing an experiment often feels like deciphering a puzzle, where incomplete documentation and missing configurations lead to inconsistent outcomes. While research infrastructures in the form of testbeds provide a structured framework for (reproducible) experiments, additional measures are necessary to completely close the loop between experiment execution, sharing of results, and re-execution. To address these challenges, we propose MARTE, a reproduction methodology for testbed-driven experiments. MARTE consists of two complementary approaches: (1) Malleable reproduction, which retains original scripts, allowing modifications for flexible reuse, and (2) automated record-replay, which automatically captures and seamlessly replays all experiment instructions. Additionally, we integrate a structured data management format to store experiment results along with all relevant metadata about the execution and environment. This new approach for packaging artifacts ensures that configurations and dependencies are preserved and remain accessible. By integrating a tool for seamless publication to open repositories, we transform experimental results into shareable artifacts that others can build upon. We argue that the barriers for researchers to achieve reproducibility must be kept as low as possible. We address this by automating key aspects of experiment documentation, re-execution, and publication. The functionality of our implementation is showcased in a demonstration experiment. Its complete artifact is openly accessible, along with an example of its reproduction and replay using MARTE.
Network path conditions, such as loss, capacity, and delay, have a significant impact on the behavior and performance of networked applications. Path property emulators are essential and widely used tools to perform evaluations under realistic conditions. However, the quality of the emulation and the potential influence on experimental results itself is rarely considered. This work highlights how the performance limitations of existing tools, such as NetEm, a network emulator based on Linux traffic control, can alter network measurements. To address these shortcomings, we introduce MoonEm, a high-performance path property emulator based on the Data Plane Development Kit (DPDK). Moreover, we present a novel approach to precisely control packet transmission times on commodity hardware. MoonEm is focused on the emulation of realistic and reproducible network conditions. Our measurements demonstrate that MoonEm achieves a maximum packet rate of 13.39 Mpps compared to 0.98 Mpps for NetEm. In contrast to NetEm, we improve latency deviation from 71.15 µs to 53 ns for the median and from 769.26 µs to 80 ns for the worst case.
Result reproducibility, a cornerstone of open science, not only enhances transparency and trustworthiness in research findings but facilitates collaboration and accelerates scientific progress by enabling other researchers to validate and build upon existing results. In computer science, testbeds are widely used research infrastructures to create scientific results. Scientists create experiments in these testbeds typically as one-off setups to answer specific questions. The outcomes of this process are individualized experiments with a wide variety of approaches, structures, formats, or documentation. These properties hamper effective experiment validation or reuse. This paper proposes a concept to structure and document the outcome of testbed-hosted experiments. We describe our implementation that packages the results of a specific testbed framework, called plain orchestrating service (pos), in the standardized RO-Crate format and annotates metadata to the result files. The metadata includes information such as authorship, affiliations, software setup, hardware setup, energy consumption, and network topology. Furthermore, we demonstrate our implementation with an example measurement and publish the packaged RO-Crate to the open repository Zenodo. Our approach enables researchers to effectively organize, share, and reproduce experimental data, promoting transparency and collaboration.
The development and roll-out of new Ethernet standards increase the available bandwidths in computer networks. This growth presents significant advantages, enabling novel applications. At the same time, the increase introduces new challenges; higher data rates reduce the available time budget to process each packet. This development also impacts software-defined networks. Their data planes need to keep up with the increased traffic rates. Nevertheless, the control plane must not be ignored; fast reaction times are necessary to handle the increased rates handled by data planes efficiently.In our work, we analyze the interaction of a high-performance data plane and different implementations for the control plane. We selected a P4 switching ASIC as our data plane. For the control plane, we investigate vendor-specific implementations and a standardized implementation called P4Runtime. To determine the performance of the control plane, we introduce a novel measurement methodology. This methodology allows measuring the delay between the initiation of rule updates on the control plane and their application on the data plane. We investigate the behavior of the data plane, its performance and non-atomicity of updates. Based on our findings, we apply different optimization strategies to improve control plane performance. Our measurements show that neglecting the control plane performance may impact network behavior due to delayed updates, but we also show how to minimize this delay and, thereby, its impact. We have released the experiment artifacts of our study including experiment scripts and measurement data.
The continuous increase of achievable data rates in computer networks is both blessing and curse. Increasing data rates enable novel applications through higher bandwidths. However, support for higher data rates requires devices to process packets reliably in an ever-decreasing amount of time per packet. In terms of software-defined networking: higher data rates call for a faster data plane. Nevertheless, the control plane must not be ignored; to faithfully react to data plane behavior, a highperformance control plane is essential. Otherwise, e.g., the data plane's state cannot be updated fast enough to cope with fastpaced traffic changes. In this case study, we investigate the control plane of a high-performance P4 switching ASIC. Moreover, we create a measurement methodology to track the delay between the reception of a rule update on the control plane and its actual application on the data plane of a P4 hardware switch. By applying the methodology to said ASIC, we can precisely describe its performance and non-atomicity in updates. Based on our findings, we apply multiple different approaches to optimize control plane latency. Our results highlight the need to consider latency on the control plane proportionate with the increase of achievable data rates.
Quantum Computers (QCs) differ radically from traditional computers and can efficiently solve mathematical problems fundamental to our current cryptographic algorithms. Although existing QCs need to accommodate more qubits to break cryptographic algorithms, the concern of ''Store-Now-Decrypt-Later'' (i.e., adversaries store encrypted data today and decrypt them once powerful QCs become available) highlights the necessity to adopt quantum-safe approaches as soon as possible. In this work, we investigate the performance impact of Post-Quantum Cryptography (PQC) on TLS 1.3. Different signature algorithms and key agreements (as proposed by the National Institute of Standards and Technology (NIST)) are examined through black- and white-box measurements to get precise handshake latencies and computational costs per participating library. We emulated loss, bandwidth, and delay to analyze constrained environments. Our results reveal that HQC and Kyber are on par with our current state-of-the-art, while Dilithium and Falcon are even faster. We observed no performance drawback from using hybrid algorithms; moreover, on higher NIST security levels, PQC outperformed any algorithm in use today. Hence, we conclude that post-quantum TLS is suitable for adoption in today's systems.
Running applications over a shared network may lead to unwanted impairments or performance impacts. To avoid these effects, the partitioning of network resources is an integral aspect of effective 5G networks. These virtually partitioned networks or slices allow the provisioning of network resources to guarantee a specific service quality to dedicated virtual networks. Programmable network devices, pushed by new languages such as P4, with their intrinsic flexibility, present themselves as a promising technique to realize slicing. This paper explores three approaches to network slicing and their respective implementation on a P4 soft- and hardware network device. We focus our effort on investigating P4 primitives that do not require the features of a specific P4 device but are available across different P4 targets. Based on our findings, we provide target-specific guidelines minimizing the impact of P4-based slicing for software and hardware targets alike.
—In scientific research, the independent reproduction of experiments is the source of trust. Detailed documentation is required to enable experiment reproduction. Reproducibility awards were created to honor the increased documentation effort. In this work, we propose a novel approach toward reproducible research—a structured experimental workflow that allows the creation of reproducible experiments without requiring additional efforts of the researcher. Moreover, we present our own testbed and toolchain, namely, plain orchestrating service (pos), which enables the creation of such experimental work-flows. The experiment is documented by our proposed, fully scripted experiment structure. In addition, pos provides scripts enabling the automation of the bundling and release of all experimental artifacts. We provide an interactive environment where pos experiments can be executed and reproduced, available at https://gallenmu.github.io/single-server-experiment.
The creation of prototypes is a convincing approach, demonstrating the feasibility of scientific ideas. Testbeds act as enablers for such prototypes, contributing the facilities to their construction. In this paper, we apply a prototype-driven approach to the development of the testbeds themselves. Thus, we select abstractions and APIs to modularize testbeds to allow a selective adaptation or substitution of specific components. To minimize costs, our approach aims to consolidate all components into a single system. Hence, we named it testbed on a single system (toast). The single-server approach demands the recreation of entire components in software such as networks or experiment nodes. Simultaneously, the softwarization of components enables flexible network topologies and scalability. At the same time, we try to keep the behavior and the performance as close to a hardware-based testbed as possible. Therefore, we rely heavily on hardware acceleration of IO using techniques such as single root IO virtualization (SR-IOV). A case study compares the accelerated IO of toast to a hardware-based testbed and a testbed without IO acceleration. We want to use toast as a training and teaching environment and a prototype facility for future research infrastructures.
To enable cooperation of cyber-physical systems in latency-critical scenarios, control algorithms are placed in edge systems communicating with sensors and actuators via wireless channels. The shift from wired towards wireless communication is accompanied by an inherent lack of predictability due to interference and mobility. The state of the art in distributed controller design is proactive in nature, modeling and predicting (and potentially oversimplifying) channel properties stochastically or pessimistically, i.e., worst-case considerations. In contrast, we present a system based on a real-time transport protocol that is aware of application-level constraints and applies run-time measurements for channel properties. Our run-time system utilizes this information to select appropriate controller instances, i.e., gain scheduling, that can handle the current conditions. We evaluate our system empirically in a wireless testbed employing a shielded environment to ensure reproducible channel conditions. A series of measurements demonstrates predictability of latency and potential limits for wireless networked control.