In a distributed system where computational tasks can be offloaded, monitoring the progress of offloaded tasks allows for more reliable operation of the network since tasks can be reassigned if not meeting expectations. Determining when to reassign the task is critical to the system, as reassigning too early can unnecessarily increase the logistical burden, while reassigning too late means more time where system performance is inadequate. We consider this problem in a detection frame work, with the goal of minimizing the amount of time-lost due to poor performance, and determine both what the optimal detector is and what the maximum time-lost should be under mild assumptions. To mimic real-world settings, we assume that performance has been bench-marked with only the moments known.

In tactical networks, applications and services depend on awareness of dynamics and uncertainty of the environment. These approaches also must deal with resource limitations of the nodes running the applications and of the limited available bandwidth for these nodes to exchange information. These systems might employ a set of applications that demonstrate a range of performance metrics (e.g., processing latency and training accuracy of deployed models). Services managing these networks must resource-efficiently maintain understanding and awareness of available resources to execute these applications. They must then make complex decisions on how and where to execute these applications. With poor understanding of the available resources, this problem is even more difficult and the performance of the system might be suboptimal. In this paper, we explore monitoring strategies that can adapt to the degree of dynamics of the environment and test out task allocation strategies that account for environmental dynamics. Through empirical study, we study the impact of monitoring strategies on compare the performance for various adaptive resource monitoring and resource allocation strategies.

In this paper, we study a goal-oriented communication problem for edge server monitoring, where compute jobs arrive intermittently at dispatchers and must be immediately assigned to distributed edge servers. Due to competing workloads and the dynamic nature of the edge environment, server availability fluctuates over time. To maintain accurate estimates of server availability states, each dispatcher updates its belief using two mechanisms: (i) active queries over shared communication channels and (ii) feedback from past job executions. We formulate a query scheduling problem that maximizes the job success rate under limited communication resources for queries. This problem is modeled as a Restless Multi-Armed Bandit (RMAB) with multiple actions and addressed using a Net-Gain Maximization (NGM) scheduling algorithm, which selects servers to query based on their expected improvement in execution performance. Simulation results show that the proposed NGM Policy significantly outperforms baseline strategies, achieving up to a 30% gain over the Round-Robin Policy and up to a 107% gain over the Never-Query Policy.

In this paper, we consider a goal-oriented communication problem for edge server monitoring, where jobs arrive intermittently at multiple dispatchers and must be assigned to shared edge servers with finite queues and time-varying availability. Accurate knowledge of server status is critical for sustaining high throughput, yet remains challenging under dynamic workloads and partial observability. To address this challenge, each dispatcher maintains server knowledge through two complementary mechanisms: (i) active status queries that provide instantaneous updates at a communication cost, and (ii) job execution feedback that reveals server conditions upon successful or failed job completion. We formulate a cooperative multi-agent distributed decision-making problem in which dispatchers jointly optimize query scheduling to balance throughput against communication overhead. To solve this problem, we propose a Multi-Agent Proximal Policy Optimization (MAPPO)-based algorithm that leverages centralized training with de-centralized execution (CTDE) to learn distributed query-and-dispatch policies under partial and stale observations. Experiments show that MAPPO achieves superior throughput-cost tradeoffs and significantly outperforms baseline strategies across varying query costs, job arrival rates, and dispatchers.

When making decisions in a network, it is important to have up-to-date knowledge of the current state of the system. Obtaining this information, however, comes at a cost. In this paper, we determine the optimal finite-time update policy for monitoring the binary states of remote sources with a reporting rate constraint. We first prove an upper and lower bound of the minimal probability of error before solving the problem analytically. The error probability is defined as the probability that the system performs differently than it would with full system knowledge. More specifically, an error occurs when the destination node incorrectly determines which top- K priority sources are in the “free” state. We find that the optimal policy follows a specific ordered 3-stage update pattern. We then provide the optimal transition points for each stage for each source.

Estimation of transmission rates by a malicious user can serve as a stepping stone to further attacks on the network. In this paper, we aim to investigate the general problem of estimating traffic transmission rates in a CSMA/CA network using a class of passive eavesdropping methods. We consider the case where a single eavesdropper passively monitors all active network nodes but cannot observe all packet transmissions due to spatial-reuse collisions. To enable tractable analysis, we first propose an approximate statistical model that can help the eavesdropper estimate transmission rates with partial measurements under spatial reuse. We next consider a class of eavesdroppers that become increasingly more capable, and develop a framework to demonstrate that two classes of eavesdropper capabilities are sufficient to achieve a consistent transmission rate estimator. We provide numerical tests of our proposed estimators under practical network cases using the NS-3 simulator that validate the theoretical results.

A variety of theoretically-sound policy gradient algorithms exist for the on-policy setting due to the policy gradient theorem, which provides a simplified form for the gradient. The off-policy setting, however, has been less clear due to the existence of multiple objectives and the lack of an explicit off-policy policy gradient theorem. In this work, we unify these objectives into one off-policy objective, and provide a policy gradient theorem for this unified objective. The derivation involves emphatic weightings and interest functions. We show multiple strategies to approximate the gradients, in an algorithm called Actor Critic with Emphatic weightings (ACE). We prove in a counterexample that previous (semi-gradient) off-policy actor-critic methods--particularly Off-Policy Actor-Critic (OffPAC) and Deterministic Policy Gradient (DPG)--converge to the wrong solution whereas ACE finds the optimal solution. We also highlight why these semi-gradient approaches can still perform well in practice, suggesting strategies for variance reduction in ACE. We empirically study several variants of ACE on two classic control environments and an image-based environment designed to illustrate the tradeoffs made by each gradient approximation. We find that by approximating the emphatic weightings directly, ACE performs as well as or better than OffPAC in all settings tested.

Importance sampling is a central idea underlying off-policy prediction in reinforcement learning. It provides a strategy for re-weighting samples from a distribution to obtain unbiased estimates under another distribution. However, importance sampling weights tend to exhibit extreme variance, often leading to stability issues in practice. In this work, we consider a broader class of importance weights to correct samples in off-policy learning. We propose the use of value-aware importance weights which take into account the sample space to provide lower variance, but still unbiased, estimates under a target distribution. We derive how such weights can be computed, and detail key properties of the resulting importance weights. We then extend several reinforcement learning prediction algorithms to the off-policy setting with these weights, and evaluate them empirically.

This work establishes that the physical layer can be used to perform information-theoretic authentication in additive white Gaussian noise (AWGN) channels, as long as the adversary is not omniscient. The model considered consists of an encoder, decoder, and adversary, where the adversary knows the message given to the encoder, has a non-causal noisy observation of the encoder’s transmission and may use unlimited transmission power, while the decoder observes a noisy version of the sum of the encoder and adversary’s outputs. A method to modify a generic existing channel code to enable authentication is presented. This method relies on injecting message-dependent noise into the transmission and accepting the transmission as authentic only if the correct noise levels for the decoded message are observed. One drawback to this method is that the encoder must still transmit a low-power signal in the case where there is no message to send. It is shown that this modification costs an asymptotically negligible amount of the coding rate, while still enabling authentication as long as the adversary’s observation is not noiseless. Also notable is that this modification is not (asymptotically) a function of the statistical characterization of the adversary’s channel and no secret key is required. We believe these features will pave the way for a robust practical implementation. Using these results, the channel-authenticated capacity is calculated and shown to be equal to the non-adversarial channel capacity. As our results will show, information-theoretic authentication in AWGN channels is possible without the need for the legitimate party to have a model-based advantage over the adversary. While this modular scheme is designed for use in the given channel model, it is applicable to a wide range of settings.

We present a new threat model enabling a passive adversary to infer which overheard packets belong to which transmitters. We call this threat model unsupervised wireless diarization (UWD) where the adversary assigns transmitter identity (label) to received packets in an encrypted wireless network without access to the MAC headers. To demonstrate the feasibility of such an attack, we develop UWDNet, a wireless diarization pipeline comprised of a Siamese neural network to extract embeddings from received packets, a similarity metric to compare embeddings, and unsupervised clustering. We evaluate UWDNet on both synthetic datasets and datasets of real wireless transmissions collected using Rice University's configurable massive MIMO testbed RENEW. Via various experimentation scenarios, our initial results show that UWDNet achieves a diarization accuracy of above 90% on synthetic data of transmitters it has never seen. To push the limits of performance evaluation, we collected a real radio transmissions dataset representing a worst-case (almost pathological) setting where all nodes are co-located. Even in this near-pathological case, UWDNet accuracy is > 60% – well above a random label assignment, indicating the feasibility of unsupervised wireless diarization in real-life scenarios. We also analyzed different factors such as the spatial channel and transmit parameters, which impact diarization accuracy in real-world scenarios.

This paper investigates the secret key-authenticated-capacity region, where information-theoretic authentication is defined by the ability of the decoder to accept and decode messages originating from a valid encoder while rejecting messages from other invalid sources. The model considered here consists of a valid encoder-decoder pairing that can communicate through a channel controlled by an adversary who is also able to eavesdrop on the encoder’s transmissions. Prior to the encoder’s transmission, the adversary decides whether or not to replace the decoder’s observation with an arbitrary one of the adversary’s choosing, with the adversary’s objective being to have the decoder accept and decode their observation to a valid message (different from that of the encoder). To combat the adversary, the encoder and decoder share a secret key. The secret key-authenticated-capacity region is defined as the region of jointly achievable message rate, authentication rate (a to be defined per symbol measure that will generally represent the likelihood that an adversary can fool the decoder), and the key-consumption rate (how many bits of secret key are needed per symbol sent). This is the second of a two-part study, with the parts differing in their measure of the authentication rate. For this second study, the probability of false authentication is considered as a function of the system state, where the system state is defined by the message being transmitted, the value of the secret key, the adversary’s channel observations, and the adversary’s (possibly stochastic) choice for the decoder’s observation. Termed the typical-authentication rate, the authentication measure considered here corresponds to an upper bound on the probability of false authentication for the majority of system states. For this measure, we derive matching inner and outer bounds for the secret key-enabled authenticated capacity region in terms of traditional information-theoretic measures. In doing so, it is shown that the typical-authentication rate and the message rate exhibit a one-to-one trade-off in the capacity region.

For a broad range of C 4 ISR applications, the ability to conduct operations in dynamic tactical network environments requires the understanding of resource availability. We propose a framework for efficiently monitoring such a complex network for the purpose of allocation of networking, computing, and analytics resources. The framework maintains awareness of a heterogeneous network, including compute platforms and networking resources, to improve the performance of analytics placement. We outline the framework comprising metric selection, compression, and scheduling. We introduce the concept of network maps, a distributed reporting method where nodes determine their own reporting schedule to maintain analytics placement quality. Next, we present simulation results that show the performance improvement in monitoring and analytics placement. Finally, we describe the monitoring architecture that we are developing to conduct emulation experiments.

This paper investigates the secret-key-authenticated-capacity region, where information-theoretic authentication is defined by the ability of the decoder to accept and decode messages originating from a valid encoder while rejecting messages from other invalid sources. The model considered here consists of a valid encoder-decoder pairing that can communicate through a channel controlled by an adversary who is also able to eavesdrop on the encoder’s transmissions. Over multiple rounds of communication, the adversary first decides whether or not to replace the decoder’s observation with an arbitrary one of the adversary’s choosing, with the goal of the adversary being to have the decoder accept and decode their observation as a valid message (different from that of the encoder). To combat the adversary, the encoder and decoder share a secret key. The secret-key-authenticated-capacity region here is then defined as the region of jointly achievable message rate, authentication rate (a to be defined per symbol measure that will generally represent the likelihood that an adversary can fool the decoder), and the key-consumption rate (how many bits of secret key are needed per symbol sent). This is the first of a two-part study, with the parts differing in their measure of the authentication rate. In this first study, the authentication rate is the exponent of blocklength-normalized exponent of the expected probability of false authentication. For this metric, we provide an inner bound which improves on those existing in the literature. This is achieved by adopting and merging different classical techniques in novel ways. Within these classical secret-key-based authentication techniques, one technique derives authentication capability from secure channel coding to send the secret key with the message, and the other technique derives its authentication capability directly from obscuring the source.

A central challenge to applying many off-policy reinforcement learning algorithms to real world problems is the variance introduced by importance sampling. In off-policy learning, the agent learns about a different policy than the one being executed. To account for the difference importance sampling ratios are often used, but can increase variance in the algorithms and reduce the rate of learning. Several variations of importance sampling have been proposed to reduce variance, with per-decision importance sampling being the most popular. However, the update rules for most off-policy algorithms in the literature depart from per-decision importance sampling in a subtle way; they correct the entire TD error instead of just the TD target. In this work, we show how this slight change can be interpreted as a control variate for the TD target, reducing variance and improving performance. Experiments over a wide range of algorithms show this subtle modification results in improved performance.

We consider the problem of estimating link packet rates in CSMA/CA networks using only eavesdropped observations from a single observer. We assume that the observer does not know the network topology and hence the contention graph indicating the interference structure. Additionally, if the eavesdropped network permits spatial reuse, then there will be collisions at the eavesdropper, leading to partial measurements. We propose a link packet rate estimation algorithm that works under the challenges mentioned above, by leveraging time reversibility of the traffic on networks with single-hop flows. We demonstrate that the estimated values converge to the true values asymptotically in the duration of the observation window.

We take the first step towards understanding the fundamental limits of blind wireless network inference performed by a distributed network of single-antenna adversary nodes. The distributed adversary nodes are assumed to be blind to the protocol parameters as well as the modulation, coding and encryption schemes used by the network being monitored. Focusing on the special case of inferring the channel access probabilities of the monitored nodes, we derive minimax bounds for blind inference. We show that blind inference is possible with similar sample complexity (asymptotically) as non-blind inference given certain network connectivity conditions are satisfied.

In unsecured communications settings, ascertaining the trustworthiness of received information, called authentication, is paramount. We consider keyless authentication over an arbitrarily-varying channel, where channel states are chosen by a malicious adversary with access to noisy versions of transmitted sequences. We have shown previously that a channel condition termed U-overwritability is a sufficient condition for zero authentication capacity over such a channel, and also that with a deterministic encoder, a sufficiently clear-eyed adversary is essentially omniscient. In this paper, we show that even if the authentication capacity with a deterministic encoder and an essentially omniscient adversary is zero, allowing a stochastic encoder can result in a positive authentication capacity. Furthermore, the authentication capacity with a stochastic encoder can be equal to the no-adversary capacity of the underlying channel in this case. We illustrate this for a binary channel model, which provides insight into the more general case.

This paper investigates the secret key authentication capacity region. Specifically, the focus is on a model where a source must transmit information over an adversary controlled channel where the adversary, prior to the source's transmission, decides whether or not to replace the destination's observation with an arbitrary one of their choosing (done in hopes of having the destination accept a false message). To combat the adversary, the source and destination share a secret key which they may use to guarantee authenticated communications. The secret key authentication capacity region here is then defined as the region of jointly achievable message rate, authentication rate, and key consumption rate (i.e., how many bits of secret key are needed). This is the first of a two part study, with the parts differing in how the authentication rate is measured. In this first study the authenticated rate is measured by the traditional metric of the maximum expected probability of false authentication. For this metric, we provide an inner bound which improves on those existing in the literature. This is achieved by adopting and merging different classical techniques in novel ways. Within these classical techniques, one technique derives authentication capability directly from the noisy communications channel, and the other technique derives its' authentication capability directly from obscuring the source.