While offline reinforcement learning provides reliable policies for real-world deployment, its inherent pessimism severely restricts an agent's ability to explore and collect novel data online. Drawing inspiration from safe reinforcement learning, exploring near the boundary of regions well covered by the offline dataset and reliably modeled by the simulator allows an agent to take manageable risks–venturing into informative but moderate-uncertainty states while remaining close enough to familiar regions for safe recovery. However, naively rewarding this boundary-seeking behavior can lead to a degenerate parking behavior, where the agent simply stops once it reaches the frontier. To solve this, we propose a novel vector-field reward shaping paradigm designed to induce continuous, safe boundary exploration for non-adaptive deployed policies. Operating on an uncertainty oracle trained from offline data, our reward combines two complementary components: a gradient-alignment term that attracts the agent toward a target uncertainty level, and a rotational-flow term that promotes motion along the local tangent plane of the uncertainty manifold. Through theoretical analysis, we show that this reward structure naturally induces sustained exploratory behavior along the boundary while preventing degenerate solutions. Empirically, by integrating our proposed reward shaping with Soft Actor-Critic on a 2D continuous navigation task, we validate that agents successfully traverse uncertainty boundaries while balancing safe, informative data collection with primary task completion.

In this work, we first prove that the separation principle holds for communication-constrained LQR problems under i.i.d. zero-mean disturbances with a symmetric distribution. We then solve the dynamic programming problem and show that the optimal scheduling policy is a symmetric threshold rule on the accumulated disturbance since the most recent update, while the optimal controller is a discounted linear feedback law independent of the scheduling policy.

The large bandwidths available at millimeter wave (mmWave) FR2 bands (24-71 GHz) and the emerging FR3 bands (7-24 GHz) are essential for supporting high data rates. Highly directional beams utilized to overcome the attenuation in these frequencies necessitate robust and efficient beamforming schemes. Nevertheless, antenna and beam management approaches still face challenges in highly mobile solutions, such as vehicular connectivity, with increasing number of bands. In this work, the concepts of spectrum mobility is studied along with antenna array management in multiple frequencies to improve beamforming under mobility. The spectrum mobility problem aims to select the optimal channel frequency and beam direction in each time slot to maximize data rate. This problem is formulated as a Partially Observable Markov Decision Process (POMDP) and Point-Based Value Iteration (PBVI) algorithm is used to find a policy with performance guarantees. Numerical examples confirm the efficacy of the resulting policy for multiple available frequency bands, even when the user mobility significantly deviates from models assumed during policy generation.

Age of Information (AoI) has become a central metric for the design of wireless update systems, especially in applications where fresh measurements support tracking, estimation, and control. Despite its popularity, the use of mean AoI or peak AoI as a surrogate for closed-loop performance is often motivated by intuition rather than by a control-theoretic derivation. This paper examines whether minimizing the mean AoI is in fact optimal for networked control systems. For scalar linear time-invariant systems with delayed intermittent updates, we show that, under state-independent scheduling policies, the infinite-horizon LQR tracking problem reduces to an optimization over the distribution of inter-scheduling intervals. The resulting objective depends on higher-order statistical moments, and in unstable or correlated regimes on exponential moments, of the inter-scheduling process rather than only on its mean. Consequently, policies with identical mean AoI can induce substantially different tracking costs. We further extend the analysis to disturbances with exponentially decaying autocorrelation and derive equivalent cost formulations that expose the role of the full interval distribution. Finally, we evaluate the theory using real vehicle trajectories from the NGSIM US-101 dataset. The empirical results match the predicted performance trends, demonstrating that mean AoI alone is insufficient for control-oriented network design.

Millimeter-wave (mmWave) frequencies promise multi-gigabit connectivity for vehicle-to-everything (V2X) networks, but face challenges in terms of severe path loss and mobility-related beam misalignment. Reliable V2X connectivity requires fast, double-directional beam alignment. However, existing methods suffer from high training overhead and limited generalization to unseen scenarios. This paper presents VIsion-based BEamforming(VIBE), a hybrid model-based, closed-loop, learning architecture for real-time double-directional mmWave beam management primed by camera sensing. VIBE fuses machine learning, model-based reasoning, and closed-loop RF feedback to balance beam-pair establishment latency with link quality. VIBE bypasses exhaustive training overhead and accelerates link establishment by leveraging camera observations to reduce the beam-search space. Lightweight beam refinement and offset tracking mechanisms adaptively refine beams in response to dynamic application requirements. VIBE is implemented and evaluated across online indoor/outdoor testbeds, public datasets, and real-time vehicular experiments, demonstrating strong generalization capabilities, making it suitable for real-time V2X communication. Comparisons with 5G NR hierarchical beamforming show that VIBE consistently maintains lower outage rates. Furthermore, VIBE outperforms state-of-the-art end-to-end ML models for beam selection when evaluated on public datasets and achieves outage rates as low as 1.1-1.4

mmWave technologies have been adopted into 5G and 6G systems. Current research primarily focuses on enhancing the quality of service and data throughput in mm Wave communications, as well as improving the accuracy of sensing from their respective functional perspectives. Consequently, this emphasis often overlooks the potential benefits of integrating communication and sensing, enhancing performance across both functionalities. This paper introduces mSAC, a novel mmWave-based system that combines sensing with communication via Orthogonal Frequency-Division Multiplexing (OFDM) signals. This approach differs from traditional mmWave sensing methods that rely on Frequency-Modulated Continuous Wave (FMCW) signals, and it substantially enhances both the accuracy and range of sensing. The mSAC system features a Joint Radar-Communication (JRC) transceiver and a communication receiver, incorporating a theoretical model that fuses sensing information from synchronous and asynchronous ends. This model is crucial for smart home applications such as controlling devices, monitoring environmental conditions, and enhancing security. Comprehensive evaluations demonstrate that mSAC achieves a 65.93% improvement in sensing accuracy and a 43.19% increase in sensing range compared to conventional methods, thereby setting a new benchmark for integrated sensing and communication systems in mmWave applications.

Aligning Large Language Models (LLMs) with human values often involves balancing multiple, conflicting objectives such as helpfulness and harmlessness. Training these models is computationally intensive, and centralizing the process raises significant data privacy concerns. Federated Learning (FL) offers a compelling alternative, but existing Federated Multi-Objective Optimization (FMOO) methods face severe communication bottlenecks as their reliance on transmitting multiple gradients to a server is unscalable for large models. We introduce FIRM (Federated In-client Regularized Multi-objective alignment), a novel algorithm that achieves both client disagreement drift mitigation and communication efficiency. In FIRM, each client locally solves a regularized multi-objective optimization problem. By directly mitigating client disagreement drift through in-client regularization, our method eliminates the need for the multi-gradient transmissions common in prior works. Consequently, clients need only to transmit a single set of adapted parameters, maintaining high communication efficiency. We prove that our algorithm converges to Pareto-stationary points and, to our knowledge, provide the first finite-time convergence guarantees for this federated multi-objective alignment setting. Empirically, we show that FIRM leads to smoother training dynamics, reduced client disagreement drift, and improved reward trade-offs compared to baselines. We further propose a method to incorporate a preference over the objectives and report empirical Pareto plots, demonstrating that FIRM can smoothly adapt trade-offs between objectives in response to specified preferences.

Reinforcement learning (RL) holds great promise for network control problems, yet its deployment in real-world systems remains limited due to the instability and unpredictability of RL policies during training. To address this challenge, we propose a two-phase conservative RL framework that combines domain expertise from classical network optimization with modern deep RL techniques. Our key idea is to initialize the learning process with a stable base policy, derived from expert knowledge, and then apply conservative fine-tuning under a Kullback–Leibler (KL) divergence constraint to safely explore improved behaviors. We apply this framework to the problem of covert multi-hop routing, where the objective is to optimize data throughput while minimizing detectability by adversaries. In Phase I, we construct a reliable base policy by imitating the back-pressure algorithm, which guarantees throughput-optimal behavior and stable queue dynamics. Phase II fine-tunes this policy to improve covert performance, as measured by the Detection Error Probability (DEP), while preserving training-time stability. Empirical evaluations on a grid network show that our method enables more reliable learning than pure RL. While pure RL (e.g., PPO) can sometimes achieve higher covert performance, it frequently suffers from large queues and collapsed throughput during training. In our experiments, our conservative RL framework reduces the worst-case training-time queue length by over 99% while maintaining comparable covert communication performance.

In Reinforcement Learning (RL), tasks with instantaneous hard constraints present significant challenges, particularly when the decision space is non-convex or non-star-convex. This issue is especially relevant in domains like autonomous vehicles and robotics, where constraints such as collision avoidance often take a non-convex form, and the state-space may be large. In this paper, we establish a regret bound of (O) over tilde ((1 + 1/tau) root log (1/tau) d(3)H(4)K), applicable to both starconvex and non-star-convex cases, where d is the feature dimension, H the episode length, K the number of episodes, and t the safety threshold for a linear MDP setting. Moreover, the violation of safety constraints is zero with a high probability throughout the learning process. A key technical challenge in these settings is bounding the covering number of the value-function class, which is essential for achieving value-aware uniform concentration in model-free function approximation. For the star-convex setting, we develop a novel technique called Objective-Constraint Decomposition (OCD) to properly bound the covering number, and resolves an error in a previous work on the constrained RL. In non-star-convex scenarios, where the covering number can become infinitely large, we propose a two-phase algorithm, NonConvex Safe Least Squares Value Iteration (NCS-LSVI), which first reduces uncertainty about the safe set by playing a known safe policy. After that, it carefully balances exploration and exploitation to achieve the regret bound. Finally, numerical simulations on an autonomous driving scenario demonstrate the effectiveness of NCS-LSVI.

Integrated Sensing and Communication (ISAC) systems unify sensing and communication functionalities on a single platform, opening avenues for innovative solutions in the mmWave spectrum. Joint Radar-Communication (JRC) represents one promising approach to realizing ISAC by integrating radar and communication functionalities on a single platform. The development of ISAC systems demands flexibility in both hardware and software to accommodate diverse experimental needs. However, existing Software-Defined Radio (SDR)-based platforms for ISAC often face limitations stemming from rigid hardware configurations and algorithmic constraints tied to SDR architectures. In this work, we present an open-source ISAC software platform, O-JRC, specifically designed to enable efficient development of experimental ISAC systems and validation of advanced algorithms under complex scenarios. A core feature of O-JRC is its layered and modular architecture, which disaggregates control logic from signal processing, facilitating seamless integration of advanced control algorithms developed in efficient programming languages. This modularity enhances development flexibility, enabling independent testing of various configurations without requiring code modifications, while also simplifying the evaluation of diverse algorithms. To demonstrate O-JRC's versatility, we implemented and tested two fundamentally different machine learning algorithms on a fully-digital 4x2 MIMO ISAC experimental platform operating at 24 GHz with a 200 MHz bandwidth: a Convolutional Neural Network (CNN)-based control algorithm and a Multi-Armed Bandit (MAB)-based reinforcement learning algorithm. These implementations highlight O-JRC's capability to support the development and experimentation of a wide range of control strategies. Comprehensive testing validated O-JRC's performance, underscoring its potential to drive innovation in the ISAC field 1.

We consider a real-time monitoring system where a source node (with energy limitations) aims to keep the information status at a destination node as fresh as possible by scheduling status update transmissions over a set of channels. The freshness of information at the destination node is measured in terms of the Age of Information (AoI) metric. In this setting, a natural tradeoff exists between the transmission cost (or equivalently, energy consumption) of the source and the achievable AoI performance at the destination. This tradeoff has been optimized in the existing literature under the assumption of having a complete knowledge of the channel statistics. In this work, we develop online learning-based algorithms with finite-time guarantees that optimize this tradeoff in the practical scenario where the channel statistics are unknown to the scheduler. In particular, when the channel statistics are known, the optimal scheduling policy is first proven to have a threshold-based structure with respect to the value of AoI (i.e., it is optimal to drop updates when the AoI value is below some threshold). This key insight was then utilized to develop the proposed learning algorithms that surprisingly achieve an order-optimal regret (i.e., O(1)) with respect to the time horizon length.

The rising traffic demand is exerting pressure on the licensed spectrum of Long Term Evolution (LTE). To address this, mobile operators are leveraging the unlicensed spectrum alongside the existing licensed spectrum. Yet, employing LTE in the unlicensed spectrum presents challenges, notably the coexistence of LTE with other unlicensed communication technologies, such as Wi-Fi. This paper introduces a novel coexistence mechanism that ensures a balanced coexistence between LTE and Wi-Fi networks operating in unlicensed bands. Our approach strategically allocates resource blocks in eNodeB to LTE users, promoting efficient and fair use of the unlicensed spectrum for both LTE users and Wi-Fi networks. We cast the challenge of wireless resource allocation and fair coexistence between LTE and Wi-Fi as a non -linear integer programming problem. Subsequently, we design a Polynomial-Time Approximation Scheme (PTAS) as an approximation algorithm to tackle this problem. Numerical assessments confirm the efficiency of our proposed mechanism in contrast to other prevailing solutions.

The Federal Communications Commission (FCC) in the United States has reallocated 45 MHz from the 75 MHz spectrum previously reserved for ITS, for the use of unlicensed devices such as Wi-Fi. On the other hand, as V2X deployment and the number of V2X applications are expected to gradually increase, the remaining 30 MHz may not be sufficient for ITS communication, resulting in options seeking to share unlicensed spectrum with Wi-Fi. This paper introduces an analytical model and decentralized spectrum sharing protocol for V2X sidelink to coexist with standardized IEEE 802.11 Wi-Fi in the 5.9 GHz unlicensed spectrum. The protocol uses both theoretical projection and historical channel usage information to ensure fair coexistence between Wi-Fi and V2X sidelink. Simulation based evaluation indicates that this protocol improves the performance of both types of communications when sharing unlicensed bands.

As the demand for vehicle-to-everything communication (V2X) band in the 5.9 GHz increases, the millimeter-wave spectrum offers alternative options in unlicensed or radar-dedicated bands with wider bandwidth. Joint radar-communication (JRC) systems emerge as a comprehensive solution to effectively utilize these bands by integrating both functions within the same waveform and hardware. In this work, we present a multiple-input and multiple-output (MIMO) JRC testbed, operating in the 24 GHz mmWave band, utilizing orthogonal frequency division multiplexing (OFDM) waveform that simultaneously carries data across all subcarriers. In particular, we develop a real-time operating, full-duplex JRC prototype with a fully-digital front-end and software-defined radios, providing enhanced flexibility and capability. Additionally, for systems with high computational power, we introduce a high-resolution range-angle processing method based on the MUSIC algorithm. Through mobile experiments with multiple targets, we showcase simultaneous data transmission and high-resolution radar processing capabilities enabled by the fully-digital MIMO architecture. By leveraging radar’s tracking capability, we propose a radar-assisted precoding approach, offering a low-complexity beamforming solution with reduced feedback overhead. Our experimental results demonstrate that the proposed precoding method achieves comparable performance compared to the conventional precoding method.

mmWave systems are integral parts of 5G+ wireless systems. Large bandwidths allocated at above 20GHz translate to large data rates otherwise unattainable at lower frequencies. However, signals in the mmWave bands require highly directional beams to overcome strong attenuation and do not propagate through objects along Line-of-Sight (LoS) paths. In this work, we study scenarios with direct LoS and reflected Non-Line-of-Sight (NLoS) paths, where the LoS paths are blocked temporarily. The so-called beam selection problem aims to choose beams to establish communication between two mmWave enabled devices and determine how long the communication with the chosen beam should last. Considering the system's state, defined as the LoS blockage, is observable in one choice of the beams but not others, we formulate the problem as a generalized case of Partially Observable Markov Decision Process (POMDP). The resulting policies result in the maximization of the reward (throughput) of the system, which are demonstrated through numerical examples.

Large-scale deployment of connected vehicles with cooperative sensing and maneuvering technologies increases the demand for vehicle-to-everything communication (V2X) band in 5.9 GHz. Besides the V2X spectrum, larger millimeter-wave (mmWave) bands at 24 and 77 GHz can be leveraged to supplement V2X communication and support high data rates for emerging broadband applications. For this purpose, joint radar-communication (JRC) systems have been proposed in the literature to enable both functions to be executed simultaneously using the same waveform and hardware. In this work, we present a software-defined multiple-input multiple-output (MIMO) JRC with orthogonal frequency division multiplexing (OFDM) for the 24 GHz mmWave band. We implement a real-time operating full-duplex JRC platform using commercial software-defined radios and custom-built mmWave front-ends, leveraging a fully digital MIMO architecture. With this setup, we demonstrate simultaneous data transmission and high-resolution radar imaging capabilities of MIMO OFDM JRC in the mmWave band.

Licensed spectrum of Long Term Evolution (LTE) is being strained with the increased traffic demand. To cope with this problem, mobile operators use the unlicensed spectrum to complement the available licensed spectrum. However, using LTE in unlicensed spectrum introduces new problems, such as the coexistence between LTE and other unlicensed communication technologies (e.g., Wi-Fi). This paper proposes a new coexistence mechanism to provide a fair coexistence between LTE and Wi-Fi networks on unlicensed bands. The proposed mechanism assigns resource blocks in eNodeB to LTE users in such a way that it provides efficient and fair usage of unlicensed spectrum for all LTE users and the Wi-Fi networks. We formulate the wireless resource allocation and fair coexistence problem between LTE and Wi-Fi as a non-linear integer programming. Then, we propose an approximation algorithm, which is a Polynomial-Time Approximation Scheme (PTAS), to solve this problem. Our numerical results validate the proposed mechanism's performance vis-a-vis other existing solutions.

Networking and Artificial Intelligence (AI) are two of the most transformative information technologies over the last few decades. Building upon the synergies of these two powerful technologies, we envision designing next generation of edge networks to be highly efficient, reliable, robust and secure. To this end, in this paper, we delve into interesting and fundamental research challenges and opportunities that span two major broad and symbiotic areas: AI for Networks and Networks for AI. The former deals with the development of new AI tools and techniques that can enable the next generation AI-assisted networks; while the latter focuses on developing networking techniques and tools that will facilitate the vision of distributed intelligence, resulting in a virtuous research cycle where advances in one will help accelerate advances in the other. A wide range of applications will be further discussed to illustrate the importance of the foundational advances developed in these two areas.

Large-scale deployment of connected vehicles with cooperative sensing technologies increases the demand on the vehicular communication spectrum in 5.9 GHz allocated for the exchange of safety messages. To support the high data rates needed by such applications, the millimeter-wave (mmWave) automotive radar spectrum at 76-81 GHz can be utilized for wideband communication as well. For this purpose, various joint automotive radar-communication (JARC) systems have been proposed in the literature to perform both functions using the same wideband waveform. However, the wideband joint waveforms encounter frequency-selectivity in both radar and communication channels due to multi-path propagation. In this paper, we address the optimal joint waveform design problem to exploit the frequency-selectivity for wideband JARC operations via orthogonal frequency-division multiplexing (OFDM) wherein subcarrier coefficients are designed for optimal power allocation and phase coding. We show that the problem is a non-convex quadratically constrained quadratic programming (QCQP) problem which is known to be NP-hard. Existing approaches to solve QCQP include semidefinite relaxation (SDR) which incurs high time complexity. Instead, we propose approximation methods to solve QCQP more efficiently by leveraging structured matrices and using convex approximations. Finally, we demonstrate the efficacy of the proposed approaches through numerical simulations.

In this paper, we study an optimal user scheduling with minimum beam alignment overhead in millimeter wave networks. The problem is posed as constrained Markov decision process (CMDP) with the goal of minimizing the average beam alignment overhead subject to the average rate constraint on each user. Under a certain assumption on the rate function of the users, by using a structural result derived from the Lagrangian formulation of the CMDP, we show that the optimal policy should keep scheduling the users that are scheduled in the previous time slot unless an abrupt change in the beam direction occurs. Using this result, the complexity of the problem decreases to polynomial in the number of users. In addition, we provide a heuristic deterministic algorithm that achieves $({1}+\epsilon)$ approximation of the optimal solution, with smaller $\epsilon $ at the cost of longer transmission interval of each user. Lastly, to deal with the case where the assumption on the rate function does not hold due to beam conflicts between the users, we consider a system model that accounts for an angular channel information. A new CMDP is formulated for the problem and a heuristic algorithm based on the age information is proposed.