Ongoing standardization efforts in 5G and Time-Sensitive Networking (TSN) aim to provide safety-critical applications with real-time communication. However, network schedules often rely on idealistic delay models that can jeopardize the validity of their guarantees. For instance, we identified scenarios where a 99.99% reliability in the inner 5G network diminishes to below 10% when looking at the end-to-end guarantees in TSN. Moreover, 5G-TSN networks remain prone to abrupt delay outliers (e.g., due to 5G line-of-sight blockage) that are difficult to predict and often do not leave enough time to reactively reconfigure the network. To overcome these challenges, we introduced Full Interleaving Packet Scheduling (FIPS) as, to the best of our knowledge, the first wireless-friendly IEEE 802.1Qbv scheduler that provides probabilistic end-to-end latency guarantees by utilizing 5G packet delay histograms. We extended this work with an $(m, k)$-firm Elevation Policy that acts as a fallback to the primary scheduler (e.g., FIPS) to uphold weakly hard real-time guarantees under network conditions that are not captured by the 5G histograms. Our work thereby makes a substantial advance towards dependable QoS guarantees in networks with probabilistic delay characteristics and abrupt delay outliers.

Ongoing standardization efforts in 5G and Time-Sensitive Networking (TSN) aim to provide safety-critical applications with real-time communication. However, 5G-TSN network schedules often rely on idealistic delay models that can jeopardize the validity of their guarantees. This paper presents an (m,k)-firm Elevation Policy to uphold a base level of weakly hard real-time guarantees (WHRT). It augments the primary schedule with a dynamic priority-driven scheme to elevate the priority of m out of k consecutive frames if they experience unexpected delays. Our evaluations demonstrate the necessity of WHRT to increase fault-tolerance against 5G delay outliers and to uphold the quality of control within a 5G-TSN networked control system. Still, only a small resource overhead is imposed during epochs where the primary schedule is valid and can serve stronger QoS guarantees. The (m,k)-firm Elevation Policy thereby yields a robust but light-weight fallback mechanism to serve applications with dependable guarantees during unstable network conditions.

6G is deemed as a key technology to support emerging applications with stringent requirements for highly dependable and timecritical communication. In this paper, we investigate 6G networks integrated with TSN and how to compensate for wireless stochastic behavior which involves a large intrinsic packet delay variation. We evaluate a 6G solution to reduce packet delay variation that is based on de-jittering. For this, we propose to use virtual timeslots for providing the required time-awareness. We discuss the benefits of the proposed solution while evaluating the impact of the timeslot size on the number of schedulable TSN streams.

Many applications of cyber-physical systems require real-time communication: manufacturing, automotive, etc. Recent Ethernet standards for Time Sensitive Networking (TSN) offer time-triggered scheduling in order to guarantee low latency and jitter bounds. This requires precise frame transmission planning, which becomes especially hard when dealing with many streams, large networks, and dynamically changing communications. A very promising approach uses conflict graphs, modeling conflicting transmission configurations. Since the creation of conflict graphs is the bottleneck in these approaches, we provide an improvement to the conflict graph creation. We present a randomized selection process that reduces the overall size of the graph in half and three heuristics to improve the scheduling success. In our evaluations we show substantial improvements in the graph creation speed and the scheduling success compared to existing work, updating existing schedules in fractions of a second. Additionally, offline planning of 9000 streams was performed successfully within minutes.

Multicast allows sending a message to multiple recipients without having to create and send a separate message for each recipient. This preserves network bandwidth, which is particularly important in time-sensitive networks. These networks are commonly used to provide latency-bounded communication for real-time systems in domains like automotive, avionics, industrial internet of things, automated shop floors, and smart energy grids. The preserved bandwidth can be used to admit additional real-time messages with specific quality of service requirements or to reduce the end-to-end latencies for messages of any type. However, using multicast communication can complicate traffic planning, as it requires free queues or available downstream egress ports on all branches of the multicast tree. In this work, we present a novel multicast partitioning technique to split multicast trees into smaller multicast or unicast trees. This allows for a more fine-grained trade-off between bandwidth utilization and traffic scheduling difficulty. Thus, schedulability in dynamic systems can be improved, in terms the number of admitted streams and the accumulated network throughput. We evaluated the multicast partitioning on different network topologies and with three different scheduling algorithms. With the partitioning, 5-15% fewer streams were rejected, while achieving 5-125% more network throughput, depending on the scheduling algorithm.

The importance of time- and mission-critical applications is increasing as industries and society are advancing in their digitalization. As such applications introduce stringent quality of service (QoS) requirements on the underlying network infrastructure, open standards like time-sensitive networking (TSN) and DetNet are being developed to provide dependable and time-sensitive communication in bridged LAN networks and IP networks, respectively. More recently, the extension to support wireless time-sensitive connectivity has received special interest due to improved network deployment flexibility with mobile devices. These efforts have led to the standardized support of TSN/DetNet in 5G, allowing the 5G system and its mobile devices to be integrated transparently into TSN/DetNet networks while conforming to their user and control plane protocols. Nevertheless, the induced packet delays- and in particular the packet delay variations (PDVs)- of 5G are significantly larger than in their wired counterparts. In this work, we illustrate that established methods to configure TSN/DetNet networks insufficiently address major runtime uncertainties, and that they would consequently result in QoS impairments or scalability issues. We therefore advocate for two systematic approaches to improve traffic shaping in this setting: first, we propose Packet Delay Correction where the PDV is corrected within the 5G system. Second, we introduce "wireless-aware" TSN scheduling to account for the packet delay characteristics of 5G. We demonstrate that wireless-aware TSN scheduling and Packet Delay Correction are complementary in their design and can be combined to provide formal end-to-end reliability guarantees at scale. In addition, being a key aspect to support wireless-aware TSN scheduling in 5G and future 6G networks, we introduce a data-driven framework to predict the packet delay characteristics of the wireless systems over a finite time horizon. Finally, we identify open research questions in wireless-aware TSN engineering and call for action on future studies and standardizations.

Deterministic real-time communication with bounded delay is an essential requirement for many safety-critical cyber-physical systems, and has received much attention from major standardization bodies such as IEEE and IETF. In particular, Ethernet technology has been extended by time-triggered scheduling mechanisms in standards like TTEthernet and Time-Sensitive Networking. Although the scheduling mechanisms have become part of standards, the traffic planning algorithms to create time-triggered schedules are still an open and challenging research question due to the problem's high complexity. In particular, so-called plug-and-produce scenarios require the ability to extend schedules on the fly within seconds. The need for scalable scheduling and routing algorithms is further supported by large-scale distributed real-time systems like smart energy grids with tight communication requirements. In this paper, we tackle this challenge by proposing two novel algorithms called Hierarchical Heuristic Scheduling (H2S) and Cost-Efficient Lazy Forwarding Scheduling (CELF) to calculate time-triggered schedules for TTEthernet. H2S and CELF are highly efficient and scalable, calculating schedules for more than 45,000 streams on random networks with 1,000 bridges as well as a realistic energy grid network within sub-seconds to seconds.

Due to the growing complexity of modern data centers, failures are not uncommon any more. Therefore, fault tolerance mechanisms play a vital role in fulfilling the availability requirements. Multiple availability models have been proposed to assess compute systems, among which Bayesian network models have gained popularity in industry and research due to its powerful modeling formalism. In particular, this work focuses on assessing the availability of redundant and replicated cloud computing services with Bayesian networks. So far, research on availability has only focused on modeling either infrastructure or communication failures in Bayesian networks, but have not considered both simultaneously. This work addresses practical modeling challenges of assessing the availability of large-scale redundant and replicated services with Bayesian networks, including cascading and common-cause failures from the surrounding infrastructure and communication network. In order to ease the modeling task, this paper introduces a high-level modeling formalism to build such a Bayesian network automatically. Performance evaluations demonstrate the feasibility of the presented Bayesian network approach to assess the availability of large-scale redundant and replicated services. This model is not only applicable in the domain of cloud computing it can also be applied for general cases of local and geo-distributed systems.

Since network delays can severely impact Networked Control Systems (NCS), both guaranteed Quality of Service (QoS) at the network level and guaranteed stability at the application level in the presence of delays are essential. The recently developed Dynamic Priority Token Bucket (DPTB) aims to meet network-level requirements through dynamic yet deterministic latency bounds that depend on the application’s sending behavior.This concept provides great potential for a combined design method with NCS controllers, which we propose in this paper. We design and implement two approaches that combine DPTB with (1) the well-known robust input/output (DPTB-RobustIO) controller to provide provable stability and (2) a novel multi-level linear quadratic regulator (DPTB-MLQR) to improve control performance.For the evaluation of the approaches, an Ethernet-based NCS with a Linux software switch and an inverted pendulum is used. This benchmark setup for fully automated evaluations of co-designs that combine network scheduling and QoS mechanisms with NCS controllers running on real hardware components is made available as an open source implementation. DPTB-RobustIO and DPTB-MLQR reduced the data rate by up to 26% and increased the control performance by 27% compared to regular token buckets with RobustIO control.

Executing complex time series computations on mobile edge devices in general and augmented reality (AR) devices in particular enables a variety of novel applications. As the complexity of such computations typically exceeds the resources of the mobile device, the task may be offloaded to a remote server. However, the resulting delay of the response substantially impacts the user experience, especially for real-time applications. To overcome this delay, we propose a novel system in which client and server exchange forecasting models instead of simple values. This allows the server to forecast future input values and precalculate the corresponding output. New results can thus be predicted in advance and, considering the communication delay, arrive just in time on the mobile device. The lightweight forecasting models not only reduce the amount of data that needs to be communicated, but also local (mobile device) computation to a minimum. Our results show that we can achieve a medium euclidean distance (MED) error of 0.477 mm for bio-mechanical simulation visualizations, even in the presence of jitter and with a round trip time of 98 ms. Compared to regular offloading, we reduce the MED by 61 % and the number of transmitted messages by 81 %, while providing a better user experience.

Free WiFi services offer Internet access at many public places like airports or in public transport vehicles. However, the experienced Quality of Service is often poor and unequal between users. While some users can run bandwidth-intensive applications such as video streaming in these networks, others cannot even perform simple web browsing. The reason for this are extensive delays, caused by network congestion of bandwidth-intensive applications, which especially impair interactive applications such as web browsing.In this paper, we propose a novel approach to improve fairness and user experience when sharing common network resources. To this end, we present a Usage-Dependent Quality of Service model (UD-QoS) that dynamically prioritizes network traffic based on the usage intensity of each participant. It shields participants with low usage intensity from congestion delays caused by participants who use the service extensively. We describe the versatile configuration possibilities of UD-QoS and present a Linux-based proof-of-concept implementation.Our evaluations with a physical testbed and popular real-world applications such as video streaming, social media and web browsing show a significant reduction of response times by up to 79% with UD-QoS while keeping processing overhead low. The benefits are particularly large for participants with low usage intensity.

In this work, we explore different combinations of techniques for an interactive, on-body visualization in augmented reality (AR) of an upper arm muscle simulation model. In terms of data, we focus on a continuum-mechanical simulation model involving five different muscles of the human upper arm, with physiologically realistic geometry. In terms of use cases, we focus on the immersive illustration, education, and dissemination of such simulation models. We describe the process of developing six on-body visualization prototypes over a period of five years. For each prototype, we employed different types of motion capture, AR display technologies, and visual encoding approaches, and gathered feedback throughout outreach activities. We reflect on the development of the individual prototypes and summarize lessons learned of our exploration process into the design space of situated on-body visualization.

Numerical simulations on mobile devices are an important tool for engineers and decision makers in the field. However, providing simulation results on mobile devices is challenging due to the complexity of the simulation, requiring remote server resources and distributed mobile computation. The additional large size of multi-dimensional simulation results leads to the insufficient performance of existing approaches, especially when the bandwidth of wireless communication is scarce. In this article, we present an optimized novel approach utilizing surrogate models and data assimilation techniques to reduce the communication overhead. Evaluations show that our approach is up to 6.5 times faster than streaming results from the server while still meeting required quality constraints.

Real-time simulation in augmented reality environments is an important area of research. The simulation model used in this paper simulates the musculoskeletal system, which has many potential applications in physiotherapy, medical education, and rehabilitation. To enable such real-time simulations in augmented reality, high-performance computing clusters are typically required, which can be costly and infeasible in terms of high network delay. This paper proposes SimEdge, a domain-specific edge computing system that enables real-time pervasive simulation in augmented reality environments on heterogeneous providers. This is done by smart offloading decisions based on a continuously updated list of resource providers. SimEdge builds on previous work using surrogate models and leverages scheduling improvements and context-aware data and task placement. Significant improvements in frame rate and input lag are achieved, as well as a reduction in energy consumption. The experimental results show that the proposed approach leads to an 8.3-fold increase in frame rate and a 65.14% reduction in input lag, as well as a 35.07% reduction in energy consumption compared to the local baseline system. These results demonstrate the effectiveness of the approach in improving efficiency and responsiveness.

Over the last decade, society and industries have undergone rapid digitization that is expected to lead to the evolution of the cyber-physical continuum. End-to-end deterministic communications infrastructure is the essential glue that will bridge the digital and physical worlds of the continuum. We describe the state of the art and open challenges with respect to contemporary deterministic communications and compute technologies— 3GPP 5G, IEEE Time-Sensitive Networking, IETF DetNet, OPC UA as well as edge computing. While these technologies represent significant technological advancements towards networking Cyber-Physical Systems (CPS), we argue in this paper that they rather represent a first generation of systems that are still limited in different dimensions. In contrast, realizing future deterministic communications systems requires, firstly, seamless convergence between these technologies and, secondly, scalability to support heterogeneous (time-varying requirements) arising from diverse CPS applications. In addition, future deterministic communication networks will have to provide such characteristics end-to-end, which for CPS refers to the entire communication and computation loop, from sensors to actuators. In this paper, we discuss the state of the art regarding the main challenges towards these goals: predictability, end-to-end technology integration, end-to-end security, and scalable vertical application interfacing. We then present our vision regarding viable approaches and technological enablers to overcome these four central challenges. In particular, we argue that there is currently a window of opportunity to establish, through 6G standardization, the foundations for a scalable and converged deterministic communications and compute infrastructure. Key approaches to leverage in that regard are 6G system evolutions, wireless-friendly 6G integration with TSN and DetNet, novel end-to-end security approaches, efficient edge-cloud integrations, data-driven approaches for stochastic characterization and prediction, as well as leveraging digital twins towards system awareness.

Event-triggered control has the potential to provide a similar performance level as time-triggered (periodic) control while triggering events less frequently. It therefore appears intuitive that it is also a viable approach for distributed systems to save scarce shared network resources used for inter-agent communication. While this motivation is commonly used also for multi-agent systems, a theoretical analysis of the impact of network effects on the performance of event- and time-triggered control for such distributed systems is currently missing. With this paper, we contrast event- and time-triggered control performance for a single-integrator consensus problem under consideration of a shared communication medium. We therefore incorporate transmission delays and packet loss in our analysis and compare the triggering scheme performance under two simple medium access control protocols. We find that network effects can degrade the performance of event-triggered control beyond the performance level of time-triggered control for the same average triggering rate if the network is used intensively. Moreover, the performance advantage of event-triggered control shrinks with an increasing number of agents and is even lost for sufficiently large networks in the considered setup.

Executing complex simulations on mobile devices such as augmented reality (AR) glasses or smartphones enables many novel pervasive applications. For instance, a physiotherapist can display muscles and bones in real-time as a visual overlay on the patient's body. The major challenge of such pervasive simulations is the complexity of the simulation, which typically exceeds the resources of the mobile device by far. Offloading computationally intensive simulations to a remote server is a promising method to enable real-time simulations on resource-constrained mobile devices without compromising the quality of the simulation results. However, the results of offloaded computations may arrive with an inevitable communication delay, which is critical for real-time simulations and also induces communication overhead. In this work, we tackle these challenges by proposing a novel approach for pervasive simulations on mobile devices. We combine a low-quality local Neural Network (NN) model on the mobile device with a high-quality NN model on a remote server, particularly taking care to integrate delayed updates from the server with the local simulation results. This distributed approach has several advantages over purely local or remote execution models: We benefit from high-quality remote results, while being robust to dynamic delays, server and network failures, and we reduce the communication overhead.

Many time-sensitive networked systems, such as networked control systems or other cyber/physical systems, require well-defined Quality of Service (QoS) with guaranteed deterministic bounds on network delay. Existing QoS models typically provide no guarantees for excess traffic beyond the traffic specified during the initial admission process. This can lead to a waste of resources when applications overcompensate during resource reservation to avoid traffic violations. In this work, we propose the Dynamic Priority Token Bucket (DPTB), a fundamentally different new QoS model. DPTB permits short-time violations without immediately dropping to best-effort guarantees for excess traffic. Instead, the priority of the application gets degraded to weaker but still deterministic guarantees. At any point in time, the application can calculate the currently guaranteed delay bounds, which depend only on its own past sending behavior, to enable application-level adaptation of the sending rate and application-side prediction of the implications onto the application performance. We designed DPTB as a token bucket extension that can be used on top of several existing scheduling mechanisms, such as the Asynchronous Traffic Shaper of IEEE Time-Sensitive Networking (TSN). Our evaluations show that DPTB is more resilient to bursty cross-traffic, resulting in significantly lower average delays than regular reservations.

Event-triggered control (ETC) and time-triggered control (TTC), the classical concepts to determine the transmission instants for networked control systems, each come with drawbacks: It is difficult to tune ETC such that a certain bandwidth is respected, whereas TTC cannot adapt the sampling interval to the current state of the control system. In this article, we provide an overview over rollout ETC, a method aimed at reconciling the advantages of ETC and TTC. We unite two variants of rollout ETC under a common framework and present conditions for convergence and compliance with a predefined bandwidth limit. Furthermore, we demonstrate that rollout ETC satisfies a performance bound and that it allows for a very flexible transmission scheduling similar to classical ETC. The mentioned beneficial properties are illustrated through extensive numerical simulations.

Many networked applications, e.g., in the domain of cyber-physical systems, require strict service guarantees for time-triggered traffic flows, usually in the form of jitter and latency bounds. It is a notoriously hard problem to compute a network-wide traffic plan, i.e., a set of routes and transmission schedules, that satisfies these requirements, and dynamic changes in the flow set add even more challenges. Existing traffic-planning methods are ill-suited for dynamic scenarios because they either suffer from high computational cost, can result in low network utilization, or provide no explicit guarantees when transitioning to a new traffic plan that incorporates new flows. Therefore, we present a novel approach for dynamic traffic planning of time-triggered flows. Our conflict-graph-based modeling of the traffic planning problem allows for the reconfiguration of active flows to increase the network utilization, while also providing per-flow QoS guarantees during the transition to the new traffic plan. Additionally, we introduce a novel heuristic for computing the new traffic plans. Evaluations of our prototypical implementation show that we can efficiently compute new traffic plans in scenarios with hundreds of active flows for a wide range of settings.