The evolution of technology (augmented reality, virtual reality, extended reality, artificial intelligence [AI], Internet of Things [IoT], virtualization, wireless communication, bandwidth increases, etc.), along with the digital transformation of the electric grid, has seen the emergence of a multitude of new use cases these recent years. The impact on the communication infrastructure is so important that the network must reinvent itself and be more performant, more predictive, and secure by design.

The wireless medium presents significant specific challenges to achieve properties similar to those of wired deterministic networks. At the same time, a number of use cases cannot be solved with wires and justify the extra effort of going wireless. This document presents wireless use cases (such as aeronautical communications, amusement parks, industrial applications, pro audio and video, gaming, Unmanned Aerial Vehicle (UAV) and vehicle-to-vehicle (V2V) control, edge robotics

This document updates RFC 8138 by defining a bit in the Routing Protocol for Low-Power and Lossy Networks (RPL) Destination-Oriented Directed Acyclic Graph (DODAG) Configuration option to indicate whether compression is used within the RPL Instance and to specify the behavior of nodes compliant with RFC 8138 when the bit is set and unset. Stream: Internet Engineering Task Force (IETF) RFC: 9035 Updates: 8138 Category: Standards Track Published: April 2021 ISSN: 2070-1721 Authors: P. Thubert, Ed. Cisco Systems L. Zhao Cisco Systems Status of This Memo This is an Internet Standards Track document. This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Further information on Internet Standards is available in Section 2 of RFC 7841. Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at . https://www.rfc-editor.org/info/rfc9035 Copyright Notice Copyright (c) 2021 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents ( ) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions https://trustee.ietf.org/license-info Thubert & Zhao Standards Track Page 1

This document describes a network architecture that provides low-latency, low-jitter, and highreliability packet delivery. It combines a high-speed powered backbone and subnetworks using IEEE 802.15.4 time-slotted channel hopping (TSCH) to meet the requirements of low-power wireless deterministic applications. Stream: Internet Engineering Task Force (IETF) RFC: 9030 Category: Informational Published: May 2021 ISSN: 2070-1721 Author: P. Thubert, Ed. Cisco Systems

Estimating channel conditions to predict packet delivery can be exploited as a powerful tool to ensure wireless networks dependability. In this article we explore the practical application of this idea from the end-device perspective, using the LoRaWAN protocol stack. We aim to understand if packet delivery can be estimated considering different levels of feedback at the end-device. For that, an extensive data collection campaign is carried out. Through an analysis of the obtained traces, we establish correlations between connectivity metrics at the end node and the fact that a packet is received at the gateway. The study is complemented considering different levels of feedback: (i) No feedback, (ii) enabling acknowledgements frames, and (iii) considering application/control plane data about the channel status at the gateway side. The results show that it is possible to estimate packet delivery in all the evaluated cases.

An industrial Internet of Things (IoT) deployment can potentially have thousands of devices. The need to perform, for example, a software upgrade on such devices presents a number of difficulties. While Bit Index Explicit Replication (BIER) offers a potential solution to some of those difficulties, with respect to low-cost IoT devices, utilizing BIER also presents various challenges. To address these challenges techniques are presented herein that facilitate the on-demand installation and uninstallation of a BIER state for the optimal replication of a multicast flow in support of, for example, software upgrades in an IoT domain, such as Routing Protocol for Low-Power and Lossy Networks (RPL) domain.

Within a Data Center (DC) environment network upgrades are often challenging and may consume significant amounts of time and network administrator resources. Additionally, DC networks tend to consume large amounts of energy and dissipate considerable amounts of heat that can be challenging to evacuate in densely populated fabrics. To address these challenges techniques are presented herein that support, possibly among other things, the construction of a network model; the use of Machine Learning (ML) to predict low and high load periods and, in low periods, determining the ratio of resources that may be taken offline; updating the equal-cost multi-path (ECMP) rules in the leaves to avoid selected planes so as to take the full plane spine and super-spine nodes offline; and upgrading one of the super-spine nodes and then one of the spine nodes to ensure that there is always a rollback path in case of a problem. If the upgrading is successful, the techniques may include proceeding to upgrade all of the super-spine nodes and all of the spine nodes. DETAILED DESCRIPTION Network upgrades are often painful and consume a significant number of cycles. Among other things, network administrators need to plan for the maintenance windows and often opt for a period of downtime. A facility such as In-Service Software Upgrade (ISSU) provides a mechanism to upgrade a network device with zero data plane down time. While ISSU can change the software version on a device it cannot change the configuration of the device. There are methods like fast reloads that help in changing the configuration of a device – e.g., after reloading the device can come up with a new configuration and the device will be up after a fixed amount of down time. Both ISSU and fast reloads assume 2 Sundaram and Thubert: MAINTENANCE AND POWER SAVINGS IN LARGE MULTIPLANE DATA CENTER FAB Published by Technical Disclosure Commons, 2020 2 6559 certain dimensions regarding the scale of an environment in connection with what can be brought up and running within a certain amount of time. While ISSU is mostly nondisruptive under certain constraints, since there is some control plane down time, the learning of new flows during this window is not guaranteed. Also, based on different router form factors the ISSU times for a top of rack (ToR) device and an end of row (EoR) device for platform upgrade times can vary. In service upgrades do not change configurations, and configuration changes can have additional down time – e.g., like fast reload where a router can be booted with a new configuration. The down times can depend on the type of configuration that is being pushed. Additionally, applying a new image to a router typically reboots the routing process. As well, setting a link offline may cause packet drops until the routing has recovered in the network. Aspects of the techniques presented herein load balance around a device (e.g., a router) before it is made offline. Aspects of the techniques presented herein employ a ML and Artificial Intelligence (AI) approach with software-defined networking (SDN) technology to understand the traffic patterns, the utilization of network components, feature dependency, etc. at different times to predict low utilization time slots for the components (e.g., nodes) in support of software upgrades for various routing planes in a DC fabric. Further aspects of the techniques presented herein employ planned traffic rerouting to approach optimal scenarios to achieve a smooth upgrade with lower or no down time. DC networks tend to consume large amounts of energy. In turn, DC systems (such as, for example, NX-OS-based systems) operating at high speed dissipate considerable amounts of heat that can be challenging to evacuate in densely populated fabrics. These characteristics arise even when the network experiences lower utilization – e.g., depending upon the time of the day or the day of the week, with special days like Christmas and Black Friday, etc. Green operations consist of shutting down some routers and routing around the failures. But a basic ‘power off’ of a router leads to a service disruption until the network converges. Therefore more subtle mechanisms are required. For instance, injecting an overload bit in an Intermediate System to Intermediate System (ISIS) protocol or 3 Defensive Publications Series, Art. 3703 [2020] https://www.tdcommons.org/dpubs_series/3703 3 6559 establishing high costs on the router's links to indicate that the router is not willing to route before it is effectively shut down (e.g., a make before break approach). Aspects of the techniques presented herein support detecting a window of opportunity where the network is less loaded, within that window reducing the load of a router to nominally zero (0), and taking the router offline either for upgrading or just to save energy. An additional challenge with the upgrade activity that was described above is a coexistence problem. That is, the new image or the new configuration of an updated router may behave differently from the old version and even though it interacts well with upgraded routers, it may not do so in a brownfield environment comprising legacy routers. Upgrades therefore create an ordering problem where upgrades must be completed in a certain order and enable rollback. The challenges that were described above are normally approached as a network problem. In a traditional routing world one may have, for example, a main path (e.g., Sender Policy Framework (SPF)), Traffic Engineering (TE) paths (e.g., SDN), and a Free Range Routing (FRR) path (e.g., Topology-Independent Loop-Free Alternate (TI-LFA)). Either of the above described methods may be used to route around a router to place it offline. However, because of the coexistence problem this is not sufficient, as there is still a problem of scheduling and rolling back which requires special attention from an operator to avoid stalled nodes in the network. DCs differ from a classical interior gateway protocol (IGP) environment. Accordingly, aspects of the techniques presented herein leverage that difference to support both new upgrade mechanisms and green operations. On one hand, a DC is a world of huge ECMP network routing strategies, where all of the leaf-to-leaf paths are basically equivalent and usable. Thus in a DC world the problem is not necessarily addressable as a routing change, but rather as a forwarding change whereby the ECMP operation is altered to avoid some routers. On the other hand, large DC fabrics are organized in planes. As one example, consider for instance Facebook's design where each plane may be depicted as a different color, as illustrated in Figure 1, below. 4 Sundaram and Thubert: MAINTENANCE AND POWER SAVINGS IN LARGE MULTIPLANE DATA CENTER FAB Published by Technical Disclosure Commons, 2020 4 6559 Figure 1: Exemplary Facebook Design From all of the above several fundamental observations include, possibly among other things, that: 1. Traffic is always leaf-in/leaf-out, the leaves being the nodes at the bottom of the schemas. 2. The ingress leaf selects the plane. 3. All nodes above the leaf level belong to a given plane (i.e., they have one single color whereas the leaves are white, combining all of the colors). 4. Routing happens within a plane all the way from ingress to egress. From these observations, aspects of the techniques presented herein leverage the partition of the cloud network into planes to selectively put to sleep and then upgrade a full plane. Elements of particular interest and note within the techniques that are presented herein are discussed below. A first element of aspects of the techniques presented herein supports the construction of a network model to predict flow patterns in a DC fabric. One activity that may take place during the construction of a network model is topology discovery. The network model will learn the link connectivity, capacity, etc. of the routers and will build a topological view. The topology will also learn the various routing planes. The 5 Defensive Publications Series, Art. 3703 [2020] https://www.tdcommons.org/dpubs_series/3703 5 6559 samples are chosen such that they reflect peaks of the various dimensions used. Data samples of link usages at different times, node usages, CPU consumption, and flow patterns may be sampled and learnt using telemetry data and are classified per routing planes in the fabric. The model is also able to predict the flow occurrences and flow patterns based on learning methods. The model will also predict the set of flows that are more likely to be impacted on a per routing plane down. The flow patterns of long flows are learnt by transporting the Elephant flow trap tables in telemetry. The model will also look at port queue utilization and congestion, the routing patterns, routing tables, etc. and track low utilization times for each node and calculate the effect of node removal or a routing change. A table may be constructed for each plane based on timestamps and a list of routers with various parameters such as, for example, congestion, bandwidth, multicast states, links, etc. as different dimensions. Minimized windows on a router or load at various times may be matched to that needed for the upgrade. Additionally, high priority flows that may be present on the router during the window may be predicted. Another activity that may take place during the construction of a network model comprises the development of a range of measurements. Such measurements may include, for example: 1. Plane and node utilization. A value that combines various relations such as, for example, CPU usage, route states, unicast flows, multicast floes, congestion on the links, and link utilization in a per plane way. In addition to arriving at a specific number, also determining how each of these various dimensions contribute to the node utilization or plane load. For a node utilization value one may identify how eac

The IETF IPv6 over the TSCH mode of IEEE802.15.4e (6TiSCH) working group has standardized a set of protocols to enable low power industrial-grade IPv6 networks. 6TiSCH proposes a protocol stack rooted in the Time Slotted Channel Hopping (TSCH) mode of the IEEE802.15.4-2015 standard, supports multi-hop topologies with the IPv6 Routing Protocol for Low-Power and Lossy Networks (RPL) routing protocol, and is IPv6-ready through 6LoWPAN. 6TiSCH has defined the missing control plane protocols to match link-layer resources to the routing topology and application communication needs. 6TiSCH has also defined a secure light-weight join processes combining link-layer security features (through Counter with CBC-MAC (CCM*)) with a secure joining procedure using the Constrained Application Protocol (CoAP). This tutorial provides a comprehensive overview of the 6TiSCH architecture and protocol suite, including the 6TiSCH Operation Sublayer (6top), the 6top Protocol (6P), and how it uses 6LoWPAN, IP-in-IP encapsulation, and RPL. This document is meant to be used both as a primer, and as a reference. It is tailored to the advanced researcher and engineer implementing and building upon IETF 6TiSCH specifications.

With the development of industry 4.0 and the recent evolution of the substation automation, as prescribed at least by the International Electrotechnical Commission (IEC) 61850 Standard, the network is becoming one of the key element of these trends. Network design and network architecture are becoming more and more complex and leading to challenging problems and issues, such as network security, multiplication of unmanaged broadcast domains, and bandwidth limitations. Recent tools have been introduced to help network engineers visualize different industrial Internet of Things (IIoT) protocol flows and characterizations for devices connected to the network. However, visualization is not enough and any help in the design and configuration of the network would be a great differentiator. Techniques herein provide for the ability to utilize sensors to build a network map of industrial and power data flows. The network map can then be used to configure different network slices with guaranteed bandwidth and flow isolation. DETAILED DESCRIPTION Network visualization tools have been developed that provide for the ability to analyze and visualize all industrial (IIot) flows in a network utilizing different protocols, such as Modbus or Fieldbus protocols. Such tools can be extended to consider substation automation protocols, such as IEC 61850 Sample Values (SVs), Generic Object Oriented Substation Events (GOOSE), Manufacturing Message Specification (MMS), Distributed Network Protocol 3 (DNP3), and/or the like. With the increasing number of devices attached to these networks, network segmentation or network slicing is a key solution that may improve network management and limit the broadcast domain. However, such network segmentation is a very complex 2 Thubert et al.: ENHANCED NETWORK SLICING FOR INDUSTRIAL AND ENERGY PROTOCOLS Published by Technical Disclosure Commons, 2020 2 6581 task that is typically by a network designer/engineer and can be a source of numerous problems. Provided herein is a solution that involves utilizing a network visualization tool to analyze industrial protocols and devices in order to draw a map of an active network. Through the map, clusters of devices can be isolated per industrial protocol. For instance, from the map, it can be determined which industrial devices subscribe to the same Fieldbus flows and the bandwidth used by such devices. In the case of IEC 61850, the analysis of SV/GOOSE flows can be utilized to develop a similar picture for substation networks. As the configuration of manufacturing chains or the design of a subsystem does not change very often, the network vision that results from such analysis is accurate. Once the mapping/analysis is performed, clusters can be further mapped to network slices with the right sizing of the bandwidth and use of microsegmentation to isolate nodes that need to talk to one another. Several methods to implement the network slicing may be used, such as virtual local area networks, (VLANs), Time-Sensitive Networking (TSN), or Committed Information Rate (CIR) mechanisms. This solution adds a concept of a Manufacturer Usage Description (MUD) microsegment, whereby an industrial node can express that it only talks to other nodes that play the roles 'a', 'b', and 'c'. With the visualization tool, the protocols that the devices speak and their roles within the protocols can be analyzed. Thus, this proposal enforces that only the roles described in the MUD form can be connected together (note that this is an abstraction that says for instance "one controller and only one controller of type X" but does not say which controller exactly). The identification of a controller can be discovered by the tool dynamically, and then the position can be occupied and the micro segment can be marked as complete and isolated. Analysis of the flows can be periodically performed to ensure that any changes in the behavior of devices can be taken into account, such that clusters and the network slicing can be modified accordingly. 3 Defensive Publications Series, Art. 3915 [2020] https://www.tdcommons.org/dpubs_series/3915 3 6581 Figure 1: Example Networking Environment Consider an example networking environment, as shown above in Figure 1. During operation, the sensors can gather all the flows going through the network and send this information to the controller that is running the network visualization tool. The controller can then computes a network topology based on VLANs and sends back the VLAN configuration to the switches. The isolation and the guaranteed bandwidth could be realized by using a TSN or CIR mechanism. One implementation for the techniques of this proposal may involve utilizing an 802.1Qbv mechanism, as illustrated below in Figure 2. 4 Thubert et al.: ENHANCED NETWORK SLICING FOR INDUSTRIAL AND ENERGY PROTOCOLS Published by Technical Disclosure Commons, 2020 4 6581 Figure 2: Example 802.1Qbv Implementation Features Using the mechanism as illustrated in Figure 2, access to the media can be partitioned in order to provide a dedicated bandwidth to a specific Queue. Such a mechanism can be implemented at each egress port and the time gate schedule can be computed to give each VLAN an associated bandwidth, such as, for example, example 10 megabits per second (Mbps) for VLAN 2 and 30 Mbps for VLAN 3. This solution can also be used with peristaltic scheduling. Either way, the solution bounds the use of the outgoing link for each "flow," which is really a slice. Thus, this solution, guarantees is a perfect isolation of the flows to provide that the flows get the same bandwidth regardless of the pressure by other flows. This is not the case with best effort queuing and fair share in which the bandwidth typically goes down with the amount of competing flows. In summary, techniques herein provide for the ability to utilize sensors to build a network map of industrial and power data flows. The network map can then be used to configure different network slices with guaranteed bandwidth and flow isolation. By 5 Defensive Publications Series, Art. 3915 [2020] https://www.tdcommons.org/dpubs_series/3915 5 6581 analyzing the dynamics of existing traffic, the approach focuses on bandwidth and isolation requirements that are automatically gathered. Such techniques also provide for an innovative implementation using network slicing based on TSN and 802.1Qbv. 6 Thubert et al.: ENHANCED NETWORK SLICING FOR INDUSTRIAL AND ENERGY PROTOCOLS Published by Technical Disclosure Commons, 2020

This document updates RFC 6775 and RFC 8505 in order to enable proxy services for IPv6 Neighbor Discovery by Routing Registrars called Backbone Routers. Backbone Routers are placed along the wireless edge of a Backbone, and federate multiple wireless links to form a single Multi-Link Subnet.

This document provides generic rules to enable the forwarding of 6LoWPAN fragment over a route-over network. Forwarding fragments can improve both the end-to-end latency and reliability, and reduce the buffer requirements in intermediate nodes; it may be implemented using RFC 4944 and virtual reassembly buffers.

Though the requirements are well defined and established, the adoption of radios for mission critical applications is lagging due to the fuzzy transmission characteristics of the wireless medium. To enable automation and safety applications, in particular for Machine‐to‐Machine traffic, strict reliability and availability demands must be met end‐to‐end. The Reliable and Available Wireless (RAW) Working Group (WG) aims specifically at enabling such guarantees by leveraging end‐to‐end Layer 3 services to compensate for the lossiness of the lower layers. In this letter, we present the Operation, Administration and Management (OAM) requirements to support industrial requirements. We detail first the operation challenge, the tools to verify that the network works properly, and to detect possibly faults. Then, we explain why we need to administrate the network to know in real time the state and the resources provisioned. Finally, we detail the mechanisms to make the network reliable and fault‐tolerant.

This document looks at different data flows through LLN (Low-Power and Lossy Networks) where RPL (IPv6 Routing Protocol for Low-Power and Lossy Networks) is used to establish routing. The document enumerates the cases where RFC6553 (RPI Option Type), RFC6554 (Routing Header for Source Routes) and IPv6-in-IPv6 encapsulation is required in data plane. This analysis provides the basis on which to design efficient compression of these headers. This document updates RFC6553 adding a change to the RPI Option Type. Additionally, this document updates RFC6550 defining a flag in the DIO Configuration option to indicate about this change and updates [RFC8138] as well to consider the new Option Type when the RPL Option is decompressed.

Packet Replication and Elimination is a method in which several copies of a data packet are sent in the network in order to achieve high reliability and low jitter. This document details how to apply Packet Replication and Elimination in RPL, especially how to exchange information within RPL control packets to let a node better select the different parents that will be used to forward the multiple copies of a packet. This document also describes the Objective Function which takes advantage of this information to implement multi-path routing.

This document updates RFC 4944 with a protocol that forwards individual fragments across a route-over mesh and recovers them end to end, with congestion control capabilities to protect the network. Stream: Internet Engineering Task Force (IETF) RFC: 8931 Updates: 4944 Category: Standards Track Published: November 2020 ISSN: 2070-1721 Author: P. Thubert, Ed. Cisco Systems Status of This Memo This is an Internet Standards Track document. This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Further information on Internet Standards is available in Section 2 of RFC 7841. Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at . https://www.rfc-editor.org/info/rfc8931 Copyright Notice Copyright (c) 2020 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents ( ) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. https://trustee.ietf.org/license-info Thubert Standards Track Page 1 Table of

This document presents a series of recent technologies that are capable of time synchronization and scheduling of transmission, making them suitable to carry time-sensitive flows with high reliability and availbility.