In Linux, numerous applications use the epoll() kernel functionality as an efficient mechanism for being notified when one out of multiple events occurs. However, as epoll() can only notify of operating system mechanisms, a costly syscall is required for notifying about an event in user space. To mitigate this, we propose HW-EPOLL, which allows generating events purely in user space. In HW-EPOLL, the generated event is written into a kernel/user space shared data structure, and the notification of a sleeping thread about the occurred event is offloaded to a hardware unit. This results in a 90

As power density becomes the main constraint of multicore systems, managing power consumption using DVFS while providing the desired performance becomes increasingly critical. Reinforcement learning (RL) performs significantly better than conventional methods in performance-power optimization under different hardware configurations and varying software applications. RL agents learn through trial-and-error by receiving rewards which is defined by an objective function (e.g. instructions-per-second (IPS)) within specified constraints (e.g. power budget). System and application requirements lead to changing objectives and constraints which in turn result in different reward functions. The RL agents adapt to these changing objectives and constraints (and hence reward functions). Equivalent-policy invariant comparison (EPIC) is a popular technique to evaluate different reward functions. EPIC provides a numerical score which quantifies the difference in two reward functions. In this work, we use this EPIC distance (score) to transfer knowledge and improve learning for changing reward functions. Experimental results using a DVFS enabled RISCV based system-on-chip implemented on an FPGA shows 16.2

Improving the sustainability and energy efficiency of compute resources in next-generation networks is crucial to cope with the ever-growing computing demand while maintaining manageable energy consumption in the processing nodes of the network infrastructure. Simultaneously, critical connected applications, such as autonomous driving, require a high level of service quality in terms of available throughput and achievable latencies. This demands considerable responsiveness from the compute resources and renders power management a challenging task. Existing solutions are not sufficiently adapted to the requirements and characteristics of such applications, making them either responsive but not very efficient, or efficient but unsuitable to provide the required service quality for critical tasks.We propose ecoNIC, a concept for energy-efficient network processing that combines an RSS-based hardware load balancer for SmartNICs with an adaptive Dynamic Voltage and Frequency Scaling (DVFS) governor. ecoNIC efficiently pins flow priorities to CPU core clusters, reducing the workload of selected cores in the process, and dynamically adjusts their clock speed to exploit freed-up capacities and save energy. We implement ecoNIC as an FPGA-prototype and integrate the DVFS governor into the Linux kernel. The experimental evaluation shows that significant energy savings can be achieved, while the employed priority-pinning ensures low tail latencies for critical traffic. Without sacrificing an increase in high-priority tail latencies, energy savings of 62% are possible. Further relaxation of the latency constraints allows for energy savings of up to 88%.

Inter-processes communication (IPC) is essential for multi-threaded applications to achieve efficient execution. Synchronization through IPC can become a bottleneck for these applications. The effectiveness of IPC is determined by both its latency and CPU utilization needed for the associated functions. Our research has revealed that for blocking IPC mechanisms, the thread scheduling functions within the Linux operating system significantly contribute to the notification latency. To address this issue, we propose a novel concept called HASIIL, which combines offloading IPC functionality with hardware-assisted scheduling to enhance IPC latency. Through this approach, we can improve the latency of blocking IPC mechanisms by up to 36% in Linux, while also improving CPU utilization by 40%.

Specialized hardware units often accelerate compute-intensive or memory-heavy functions. In previous publications, we proposed concepts to assist Linux with a hardware unit for managing waiting threads to improve blocking inter-process communication (IPC) mechanisms. This paper assesses the effectiveness of this hardware support on a Zynq platform. Although main memory accesses by our hardware unit are time-consuming, a consumer-producer application achieved an up to 220% increased message rate.

Efficient thread synchronization primitives are crucial in modern computer systems for the performant execution of interdependent code segments. In Linux, the futex() syscall is used to construct blocking synchronization primitives such as mutexes or conditional variables. When using futex, the uncontended case is efficiently handled entirely in user space. In the event of contention, the kernel is called to put the waiting thread to sleep until the state of the primitive changes to uncontended. The kernel must be notified of this change by a futex() syscall to wake-up the sleeping thread. This syscall must be issued by the thread that changes the primitive, which is a significant burden on this thread. To remove this burden, we introduce HW-FUTEX to offload the futex wake functionality to a hardware unit (HW Unit) that asynchronously initiates wake-ups of the sleeping threads. This reduces the time required to issue the futex wake functionality by at least 90% to 350 cycles, with no additional overhead in the uncontended case.

The fast pace at which new online services emerge leads to a rapid surge in the volume of network traffic. A recent approach that the research community has proposed to tackle this issue is in-network computing, which means that network devices perform more computations than before. As a result, processing demands become more varied, creating the need for flexible packet-processing architectures. State-of-the-art approaches provide a high degree of flexibility at the expense of performance for complex applications, or they ensure high performance but only for specific use cases. In order to address these limitations, we propose FlexCross. This flexible packet-processing design can process network traffic with diverse processing requirements at over 100 Gbit/s on FPGAs. Our design contains a crosspoint-queued crossbar that enables the execution of complex applications by forwarding incoming packets to the required processing engines in the specified sequence. The crossbar consists of distributed logic blocks that route incoming packets to the specified targets and resolve contentions for shared resources, as well as memory blocks for packet buffering. We implemented a prototype of FlexCross in Verilog and evaluated it via cycle-accurate register-transfer level simulations. We also conducted test runs with real-world network traffic on an FPGA. The evaluation results demonstrate that FlexCross outperforms state-of-the-art flexible packet-processing designs for different traffic loads and scenarios. The synthesis results show that our prototype consumes roughly 21% of the resources on a Virtex XCU55 UltraScale+ FPGA.

In recent years, several approaches have proposed dynamic voltage-frequency scaling (DVFS) controllers for power management in multiprocessor system-on-chips based on reinforcement learning (RL) to cope with changing system dynamics. Those implemented in software cannot respond to short-term changes in processor load, which occur at a sub-millisecond scale. On the other hand, modern processors contain various built-in hardware features, like prefetchers, reacting to short-term variations to improve system performance. Previously, we integrated such an RL-based DVFS algorithm into the hardware and applied it to optimize the performance of best-effort tasks. In this work, we augment it with preemptive shielding and eligibility traces to employ it on deadline-bound quality of service (QoS) tasks. The algorithm takes advantage of short-term variations to minimize power usage while ensuring the application’s QoS. Additionally, our algorithm complies with the power constraints necessary in mixed-critical systems unlike state-of-the-art algorithms. We demonstrate the features of our algorithm in a hardware-in-the-loop simulation by running LLVM’s single-source benchmarks on SparcV8 processors.

Future automotive architectures are expected to transition from a network-centric to a domain-centered architecture featuring central compute units. Powerful domain controllers or smart sensors alleviate the load on these central units and communication systems. These controllers execute tasks with varying criticalities on heterogeneous multicore processors, and are ideally capable of dynamically balancing the computing load between the central unit and sensors. Here, Artificial Intelligence (AI) capabilities play a crucial role, as it is in high demand for such an automotive architecture. However, AI still requires specialized accelerators to improve their computation performance. Task-oriented distributed computing with criticalities up to ASIL-D necessitates the development and utilization of specialized methodologies, such as safety, through the isolation and abstraction of low-level hardware concepts. Meanwhile, online monitoring and diagnostics become vital features to detect errors during operation. The EMDRIVE architecture includes methods, components, and strategies to enhance the performance, safety, and security of such distributed computing platforms. The nationally funded EMDRIVE project connects its twelve partners from academia and industry and is currently in its intermediate stage.

As the world becomes more connected and new digital services emerge at a fast pace, the amount of network traffic increases rapidly. Consequently, processing requirements become more varied and drive the need for flexible packet-processing designs, especially as in-network computing gains traction. Traditional approaches deploy hardware accelerators in a pipeline in the sequence that the associated tasks are supposed to be executed. Hence, they do not accommodate flows with different processing requirements and provide no possibility to remap flows to task sequences in runtime. In order to address these limitations, we propose FlexRoute, a fast, flexible and priority-aware packet-processing design that can process network traffic at a rate of over 100 Gbit/s on FPGAs. Our design consists of a reconfigurable parser and several processing engines that are arranged in a pipeline. The processing engines are equipped with processing units that execute specific tasks, flexible forwarding logic and priority-aware queuing/scheduling logic. We implement a prototype of FlexRoute in Verilog and evaluate it via cycle-accurate register-transfer level simulations. We also synthesize and implement our design on the Alveo U55C High Performance Compute Card and show its resource usage. The evaluation results demonstrate that FlexRoute can process packets of arbitrary size with different processing requirements at a traffic rate of about 70 Gbit/s significantly faster than two state-of-the-art flexible packet-processing designs.

In next-generation automotive, industrial, data center, and other mixed-critical networks, Ethernet is expected to power the backbone interconnect among multi-core compute nodes. On attached Network Interface Cards (NICs) Receive Side Scaling (RSS) supports the CPU in balancing workloads across cores for reduced tail latencies. However, state-of-the-art solutions are primarily designed for performance and less for energy-efficiency which will play an equally important role. For this reason we present ecoNIC, an RSS-based hardware load balancer for SmartNICs, and an agile Dynamic Voltage and Frequency Scaling (DVFS) governor, for energy-saving network processing. ecoNIC efficiently pins flow priorities to CPU core clusters, reducing the workload of select cores in the process, and dynamically adjusts their clock speed to exploit freed-up capacities and save energy. Within a cluster, it proactively redirects packet bursts of priority-separated flow bundles among available cores, or offloads them to neighbor nodes, once local resources tend to become highly loaded. The per-core energy consumption this way is reduced at the expense of low priority packet latencies, while high priority service qualities are maintained. Experimental evaluations applying real-world network traces yield energy savings of up to 37.9% at an increase from 559 mu s to 3.06 ms in low priority end-to-end tail latency compared to an even workload distribution without frequency scaling.

Learning classifier tables (LCTs) are lightweight, classifier based, hardware implemented reinforcement learning (RL) building blocks which enable self-adaptivity and self-optimization properties in multicore systems. LCTs are deployed per-core to learn and optimize potentially conflicting objectives and constraints. Experience replay (ER) is a replay memory technique in RL, where agents experiences are stored in a buffer and are used to improve the learning process. Implementing an ER buffer in hardware requires memory and is expensive. We introduce LCT-DER: LCT with dynamic-sized experience replay, where the classifier population and experiences share the same memory by exploiting the concept of macro-classifiers. LCT-DER performing DVFS achieves 44.5% and 4.5% lower number of power budget overshoots and IPS difference compared to a standard LCT without requiring additional memory.

The increasing complexity and unpredictability of emerging applications makes it challenging for multi-processor system-on-chips to satisfy their performance requirements while keeping power consumption within bounds. In order to tackle this problem, the research community has focused on developing dynamic resource managers that aim to optimize runtime parameters, such as clock frequency, voltage and task mapping. There is a large diversity in the approaches proposed in this context, but a class of resource managers that has gained traction recently is that of reinforcement learning-based controllers. In this paper we propose CoLeCTs, a resource manager that enhances the state-of-the-art resource manager SOSA by employing a joint reward assignment function and enabling collaborative information exchange among multiple learning agents. In this manner we tackle the suboptimal determination of local performance targets for heterogeneous applications and allow cooperative decision making for the learning agents. We evaluate and quantify the benefits of our approach via trace-based simulations.

Les latences d’accès aux données et les goulots d’étranglement de la bande passante représentent des facteurs limitatifs majeurs pour l’efficacité de calcul des architectures multi-cœurs et pluri-cœurs. Ce chapitre porte sur deux approches visant à garantir que les données à traiter et les entités de calcul restent limitées dans l’espace : la cohérence de cache par région et l’accélération proche-mémoire.

Data centers have been struggling to provide the necessary processing capacity to handle the surging rate of network traffic that is generated in an increasingly connected and service-oriented world. As a result, SmartNICs play an even more important role than before as they can offload various network applications and hence free CPU resources for application-layer processing, increase performance and reduce processing time. However, they often do not support flows with different offload requirements and cannot dynamically allocate offloads in run-time. In order to address these limitations, we propose FlexPipe, a fast, flexible and scalable packet-processing architecture for high-performance SmartNICs. Our design enables low-latency and runtime-reconfigurable packet forwarding at high traffic rates with minimal area overhead. Furthermore, it provides load-aware packet steering toward multiple offload units of the same type for low-bandwidth offloads. We implement a prototype of FlexPipe in Verilog and validate it via cycle-accurate register-transfer level simulations. Our evaluation results show that FlexPipe can process packets of arbitrary size with different offload requirements at line rate and on average 1.9x faster than a SmartNIC with a predefined sequence of offloads and 1.8x faster than PANIC, a flexible state-of-the-art SmartNIC.

Future connected applications require distributed processing on different layers in the network to orchestrate and optimize their behavior. This necessitates a change in the processing architecture, especially regarding the ability to self-adapt to changing circumstances, such as environmental changes or fault scenarios. Self-adaptivity in computing systems was previously often achieved with the MAPE-K concept. However, in real-time and low-latency environments with a distributed, hierarchical processing architecture, the reaction time of the MAPE-K cycle can be limited by its complexity. In this paper, we propose a possible solution to a quicker reaction by introducing X-MAPE: extending self-adaptivity with predefined, configurable, reflexive actions. In essence, X-MAPE aims to construct a low-latency path between monitoring the system and executing actions to influence it, to reflexively react to a changing system state and improving the system's behavior until a more optimal decision can be made in a less time-critical manner. We present possible realizations of this principle on different layers and components of future 6G networks, providing a vision and framework for adaptive computing systems with reflexive reactions.

Next-generation automotive networks will be characterized by a high number of interconnected sensors, actuators and applications on electronic control units communicating with each other over a high-speed Ethernet backbone network. As these applications have various criticalities, high volumes of fluctuating traffic with different priorities will have to be processed in a reliable and efficient manner. To cope with these challenges, we present Priority-aware Inter-Server Receive Side Scaling (prioRSS), a new SmartNIC-based hardware accelerator designed for automotive compute nodes. prioRSS builds upon Receive Side Scaling and introduces priority-awareness into an intra- and inter-node load balancer. It uses a priority-partitioned indirection table within which flows of the same priority are bundled. Low-latency reconfigurations issued by a Network Health Monitoring software allow for adapting the table content to changing network conditions. Simulative evaluations and comparisons to a priority-unaware version of our design show that prioRSS enables per-priority resource assignments without degrading end-to-end packet latencies while using the same table memory space. Paired with a priority-aware scheduler, end-to-end latencies of high priority flows can be notably reduced compared to average packet latencies, at the expense of lowest priority traffic. The best results are acquired when partitioning the table proportionally to the associated traffic share.

Learning classifier tables (LCTs) are lightweight and classifier-based reinforcement learning building blocks in hardware that inherit the concepts of learning classifier systems. LCTs are deployed to learn and optimize potentially conflicting objectives and constraints e.g., achieving a desired performance target (IPS) under a constraint (power budget). These are reflected by the reward function which provides rewards for the LCT. A supervisor provides LCTs with the objectives/targets and constraints by translating application requirements and system policies for the LCT. The supervisor combined with the LCTs enable self-adaptive and self-optimization properties in a system-on-chip. The objectives and constraints for the LCTs are changed in runtime by the supervisor resulting in different reward functions. The LCT has to learn these different reward functions by trial and error. The rate at which the LCT learns about these changes can be accelerated by applying transfer learning. LCS systems are represented by a population of classifiers. Each classifier is made up of condition, action and bookkeeping parameters like prediction, error, accuracy, experience and numerosity which directly influence the performance of an LCT. In this work, we propose LCT-TL: LCT with transfer learning, where we propose two strategies that selectively transfer the bookkeeping parameters for a change in target/constraint. We evaluate the strategies by deploying LCTs as DVFS controllers for performance-power optimization. Experimental results show that the LCT-TL performs significantly better than the state-of-the-art LCT and a tabular Q-learning agent. LCT-TL doesn’t require any additional hardware resources and negligible soft-ware overhead.

The performance of multi-threaded applications relies on efficient inter-process communication. One common practice is putting a thread asleep while waiting for a certain condition. Exemplary Linux kernel mechanisms that use this practice include futex, sockets, epoll, eventfd and pipe. Once the condition is met, i.e., the associated event has occurred, the waiting thread is notified. Optimizations for event notification mechanisms in Linux mostly target the thread which receives events. Contrarily, we identified high potential in relieving the event-generating thread and propose HAWEN, a hardware accelerator for thread wake-up support. HAWEN has been integrated into Linux event notification in a minimally intrusive manner. Gem5-based multi-core architecture simulations revealed up to 80% faster thread wake-up times and a 53% shorter event-generating syscall.

Profiling and tracing mechanisms are vital to identify performance bottlenecks for the optimization of modern multi-threaded applications [1]. While profiling generates a summary of averaged performance metrics, tracing provides a full record (trace) of timestamped events [1]. Hence, tracing is mandatory for detailed timing analysis but is usually difficult to implement. First, the generated traces can get large which usually results in a high overhead to record them. Second, lossless traces (no events get lost) with precise timestamps for correct event ordering across large multi-core systems are challenges for common tracing mechanisms. As a complementary approach to existing tracing mechanisms, we introduce GLS Tracing (Gem5-based Low-intrusive Software Tracing), which overcomes the mentioned challenges. GLS Tracing combines the flexibility and fine granular resolution of software tracepoints (TP) with the low-intrusive system analysis capabilities of the Gem5 simulator. Using new Gem5 pseudo-instructions (Trace Point Operation, TPOP), the time-consuming recording of a native TP in software tracing is offloaded to the Gem5 Full-System simulator, thus reducing the number of CPU cycles to be simulated for this tracing. This new tracing capability is integrated into the native Linux kernel software TP infrastructure TRACE_EVENT, resulting in an average overhead reduction of 93% per recorded TP (from on average 2300 clock cycles down to 160 clock cycles). In addition, a less flexible but almost non-intrusive TP is defined that adds only 5 to 10 cycles overhead per TP. This provides a lossless low-intrusive tracing solution for Gem5, which can be used in both user-space and kernel-space functions.