Inter-agent communication serves as an effective mechanism for enhancing performance in collaborative multi-agent reinforcement learning (MARL) systems. However, the inherent communication latency in practical systems induces both action decision delays and outdated information sharing, impeding MARL performance gains, particularly in time-critical applications like autonomous driving. In this work, we propose a Value-of-Information aware Low-latency Communication (VIL2C) scheme that proactively adjusts the latency distribution to mitigate its effects in MARL systems. Specifically, we define a Value of Information (VoI) metric to quantify the importance of delayed messages on the recipient agent's decision. We then design a VoI aware resource allocation method that dynamically prioritizes message transmission based on each delayed message's importance. Moreover, we propose a progressive message reception mechanism to adaptively adjust the reception duration based on received messages. We derive the optimized VoI aware resource allocation and theoretically prove the performance advantage of the proposed VIL2C scheme. Extensive experiments demonstrate that VIL2C outperforms existing approaches under various communication conditions. These gains are attributed to the low-latency transmission of high-VoI messages via resource allocation and the elimination of unnecessary waiting periods via adaptive reception duration.

Multidevice cooperative wireless sensing offers a promising solution for human motion recognition, owing to its superior privacy preservation and robustness. In the sensing process, devices continuously extract features from channel echoes and transmit them to a fusion center for motion recognition over successive time slots. The intertwined sub-processes of sensing and communication jointly determine recognition accuracy, yet simultaneously compete for limited radio resources. Moreover, the dynamic nature of practical environments further complicates this interplay due to the presence of moving interference sources and time-varying number of cellular users sharing the available bandwidth. Therefore, it is of paramount importance to jointly optimize sensing and communication resource allocation among devices and across time slots, while meticulously accounting for the impacts of dynamic environment to maximize recognition accuracy. In this paper, we propose a task-oriented integrated sensing and communication (ISAC) system for multidevice cooperative wireless motion recognition in dynamic environments. Specifically, we formulate a joint sensing and communication resource allocation problem to maximize recognition accuracy, represented by a discriminant gain metric that explicitly accounts for both sensing quality and communication constraints. Since this problem is a fractional program, we transform the original sum-of-ratios objective function into an equivalently subtractive form that facilities the development of a two-step iterative offline optimization (TSIO) algorithm to achieve the benchmark performance. Furthermore, to effectively cope with dynamic environmental influences, we further design a multi-agent reinforcement learning (MARL)-based online optimization (MRLO) scheme, which predicts environmental conditions at the subsequent time slot and adaptively optimizes resource allocation. Extensive numerical results illustrate that the proposed algorithm significantly enhances the recognition accuracy with dynamic environment influences, compared to existing benchmark algorithms. It is also observed from results that the sensing performance primarily drives recognition accuracy when energy is limited, whereas communication performance becomes the dominant factor under bandwidth constraints.

Foundation models place language throughout embodied agents, but its presence does not show what it contributes or how well that contribution is grounded. This survey separates these two questions. We define five non-exclusive functional roles for language: Specification, Embodied Representation, Action Orchestration, Grounding Regulation, and Execution Coupling. For each role, we trace the path from linguistic content to its embodied consumer and identify the observations or interventions that can test the claimed responsibility. Applying this framework to the reviewed literature reveals a recurring gap between functional use and evidential support. Interpretable or revised linguistic intermediates may be incorrect, go unused, or fail to affect later behavior. Even when actions are directly conditioned on language, system-level success does not by itself isolate language's contribution. We therefore evaluate grounding claim by claim, asking whether the reported evidence supports the specific responsibility assigned to language. Using role claims rather than architectures as the unit of comparison allows us to compare modular and end-to-end embodied agents without extending conclusions beyond the reported evidence.

Wi-Fi-based gesture recognition, driven by deep learning, holds significant promise for privacy-preserving and all-weather sensing. However, current methods typically rely on large amounts of labeled data, and Wi-Fi signals vary significantly across gestures, leading to severe performance degradation when models encounter unseen gestures. To address these challenges, we explore the potential of transferring knowledge from large pre-trained language models to improve the generalization of Wi-Fi-based gesture recognition systems. To this end, we propose a zero-shot gesture recognition framework, named Wi-CLIP. Inspired by the vision-language pre-training model CLIP, our method constructs a cross-modal radio frequency-text model centered on aligning Wi-Fi signals with textual semantics. Specifically, we develop a novel Wi-Fi signal encoder and a BERT-based text encoder, aligning the two modalities within a shared semantic space using contrastive learning. Our framework achieves an average recognition accuracy of 89.12

Agent-assisted memory recall is one critical research problem in the field of human-computer interaction. In conventional methods, the agent can retrieve information from its equipped memory module to help the person recall incomplete or vague memories. The limited size of memory module hinders the acquisition of complete memories and impacts the memory recall performance in practice. Memory theories suggest that the person’s relevant memory can be proactively activated through some effective cues. Inspired by this, we propose a novel strategy-guided agent-assisted memory recall method, allowing the agent to transform an original query into a cue-rich one via the judiciously designed strategy to help the person recall memories. To this end, there are two key challenges. (1) How to choose the appropriate recall strategy for diverse forgetting scenarios with distinct memory-recall characteristics? (2) How to obtain the high-quality responses leveraging recall strategies, given only abstract and sparsely annotated strategy patterns? To address the challenges, we propose a Recall Router framework. Specifically, we design a 5W Recall Map to classify memory queries into five typical scenarios and define fifteen recall strategy patterns across the corresponding scenarios. We then propose a hierarchical recall tree combined with the Monte Carlo Tree Search algorithm to optimize the selection of strategy and the generation of strategy responses. We construct an instruction tuning dataset and fine-tune multiple open-source large language models (LLMs) to develop MemoCue, an agent that excels in providing memory-inspired responses. Experiments on three representative datasets show that MemoCue surpasses LLM-based methods by 17.74% in recall inspiration. Further human evaluation highlights its advantages in memory-recall applications.

This study introduces a novel approach for preventing overexertion in home fitness through fine-grained detection of respiratory parameters. To overcome the robustness limitation associated with using a composite signal for wireless sensing, we introduce an optimization-based signal separation model. This model effectively disentangles composite signals into static and dynamic components, while preserving the intricate details of target movements or activities. Specifically, by constructing a reference signal derived from the dominant static component, we eliminate time-varying phase shifts and leverage the invariant property of the dynamic component's amplitude for precise separation. A system called FinerSense is developed, which is able to accurately and robustly detect fine-grained respiratory parameters such as respiration rate, depth, and inhalation-to-exhalation ratio with accuracy rates exceeding 97%, 95%, and 91%, respectively. Extensive experiments show that the developed system outperforms state-of-the-art baselines significantly, empowering users to optimize exercise intensity and duration while mitigating the risk of overexertion. We believe that this work is able to facilitate the seamless transition of wireless sensing systems from laboratory prototypes to practical and user-friendly applications.

In recent years, WiFi-based tracking has gained significant attention owing to its non-invasive, cost-effective, and ubiquitous coverage. These systems eliminate the need for wearable devices, making them highly suitable for applications in smart homes, health monitoring, and security surveillance. However, existing WiFi-based tracking systems face notable challenges, particularly in complex environments. Single-transceiver systems often rely on the estimation of Angle of Arrival (AoA) and Time of Flight (ToF), which is limited by the number of antennas and available bandwidth. Alternatively, multi-device systems are exploited to estimate the target's velocity via Doppler Frequency Shift (DFS). For the multidevice cooperative tracking systems, it is essential to evaluate the velocity estimation quality across devices and effectively leverage their complementarity to improve tracking performance. To this end, we propose AdaptTrack, an innovative human tracking system utilizing commercial WiFi devices. Specifically, we derive a quantization analysis of DFS estimation errors from Channel State Information (CSI) quotient. Furthermore, we design an adaptive device selection strategy that jointly considers the velocity estimation performance and the complementarity of WiFi devices to optimize tracking accuracy. In addition, we implement a prototype system based on commercial WiFi devices. These innovations enable AdaptTrack to achieve the high-precision tracking in complex scenarios. Extensive real-world experiments demonstrate the advantage of AdaptTrack in various environments, compared to the baselines. These results highlight its robustness, scalability, and potential as a practical solution for privacy-friendly human tracking in intelligent environments.

Traditional liquid detection approaches are often time-intensive and invasive, typically requiring the opening of containers for examination. While recent initiatives have proposed several innovative solutions, including camera-based and vibration sensor-based techniques, these approaches still face limitations in terms of convenience. The development of radio frequency (RF) technology, particularly millimeter-wave (mmWave) radar, offers a promising solution for non-invasive and contactless liquid detection. In particular, during the past few years, a number of radar-based sensing systems have been developed to detect or identify liquids. However, little work has been done on the simultaneous detection of multiple liquids. To fill this gap, we design a novel composite reflection model, which overcomes the detection challenges due to composite interference and environmental reflections, by utilizing the consistency and uniqueness of the reflection signals from multiple liquid targets. Based on the proposed model, we develop a system named MultiScanner, which is able to detect different types of liquids in multi-target scenarios, exhibiting high location independence without the need for extensive data training. Extensive experiments validate the effectiveness of MultiScanner, achieving up to 95.91% accuracy in detecting 10 hazardous-normal liquid combinations in 2-target scenarios. Moreover, even in more complex 5-target scenarios, an detection accuracy of 86.49% can be obtained. To the best of our knowledge, this is the first study that uses RF signals for multi-liquid detection.

This paper focuses on optimizing the Wi-Fi sensing ability for non-line-of-sight (NLOS) environments. Existing Wi-Fi Channel State Information (CSI) based methods are categorized into pattern-based and model-based approaches. Pattern-based methods, relying on machine learning, require extensive training data and are less adaptable to changing scenarios. Conversely, model-based methods utilize physical principles and are more robust, demanding less training data. While most current model-based research targets line-of-sight (LOS) scenarios, effective models for NLOS environments are lacking, where both reflection and refraction are significant. This study introduces the CPR model to quantify the squeezing and stretching effects of Fresnel zones in NLOS scenarios, which enhances spatial resolution. The PASTLBO algorithm, a parameter-adaptive teaching and learning-based optimization method, is proposed to optimize the deployment of Wi-Fi sensing systems by guiding transmitter and receiver placement for optimal sensing performance. Experimental results show significant improvements in sensing performance, with optimized placement of transmitters and receivers enhancing sensing performance by over 30% compared to standard placements. The study concludes that understanding and quantifying the squeezing and stretching effects in Fresnel zones can significantly improve the accuracy and reliability of Wi-Fi sensing systems in complex NLOS environments.

With the increase of China's elderly population year by year and the increasing number of elderly people living alone, the problem of elderly people death caused by falls becomes more severe. Conventional vision based approaches suffer from some privacy issues to the elderly or guardians. And a wearable sensor-based fall detection methods would leave the elderly constrained and stressed. Therefore, the fall detection technology based on WiFi wireless network has been proposed. This is owing to its significant advantages that it will not cause privacy disclosure problems, nor will it give the elderly any substantial physical bondage and pressure. In this paper, we construct a Channel State Information (CSI) acquisition platform based on regular communication packets with commercial WiFi to perform the sensing via the integration of sensing and communication. In particular, the WiFi works in the commuication mode and sends the comunication packet. The CSI is obtained from the communication package and is exploited for sensing. Then by pre-processing the collected CSI, the feature classification information related to falls is extracted for model identification. Based on the features, the neural network based fall detection algorithm is performed to realize the fall detection. The fall detection system equipped with the front-end interface based on Vue3 and the back-end service based on Python is implemented. The experiment results demonstrate that the system can achieve real-time and accurate fall detection, even when WiFi works in the communication mode.

Grain moisture sensing plays a critical role in ensuring grain quality and reducing grain losses. However, existing commercial off-the-shelf (COTS) grain moisture sensing systems are either expensive, inconvenient or inaccurate, which greatly limit their widespread deployment in real-world scenarios. To fill this gap, we develop a system called GrainSense which leverages COTS Wi-Fi devices to detect the grain moisture without the need for dedicated sensors. Specifically, we propose a wireless grain moisture detection model based on the refraction phenomenon of Wi-Fi signals and the Multiple-Input-Multiple-Output (MIMO) technology. On one hand, we correlate the grain moisture with the phase difference between two refracted Wi-Fi signals that propagate along different paths, based on which grain moisture can be deduced accordingly. On the other hand, to reduce the multi-path interference in indoor environments (e.g., the granary), we adopt Wi-Fi beamforming to enhance the refracted signal. In particular, a new signal feature (i.e., the Wi-Fi CSI beamforming ratio) is designed to eliminate the effect of sub-carrier frequency bias and cumulative phase bias. To validate the effectiveness of the developed system, we conduct extensive experiments with different types of grains in both the laboratory and the granary. Results show that the system can accurately estimate the grain moisture with an mean absolute error smaller than 5%, which meets the requirements for commercial usage. To the best of our knowledge, this is the first model-based work that achieves accurate grain moisture detection based on wireless sensing.

During the last decade, there have been lots of efforts on wireless sensing using Wi-Fi signals, which can be divided into two categories, i.e., the pattern-based approach and the model-based approach. Recently, more and more attention has been paid on the model-based approach, mainly due to its superiority of no need for collecting a large dataset or retraining the model for new environments. However, existing models are mainly designed for line-of-sight (LoS) scenarios, which are not applicable to non-line-of-sight (NLoS) scenarios, such as through-wall sensing. To bridge this gap, we put forward a through-wall wireless sensing model to reveal the sensing mechanism of Wi-Fi signals in NLoS scenarios. In particular, a refraction-aware Fresnel zone model is developed by taking into account both the reflection propagation and the refraction propagation of Wi-Fi signals. For the first time, we discover that the geometric distribution of Fresnel zones becomes uneven, due to the difference in dielectric constants between the air and the wall. Specifically, some areas become denser and other areas become sparser, leading to the squeeze effect and stretch effect of Fresnel zones. Inspired by the insight, we further put forward a new metric named compression-ratio to quantify the through-wall sensing capability of Wi-Fi signals. Meanwhile, a set of algorithms are developed to guide the deployment of Wi-Fi sensing systems. To validate the proposed model, we implement a through-wall respiration sensing prototype system. Experiments show that the respiration detection performance varies significantly when the user locates in different areas. Specifically, for two sensing locations (one in the compression area and the other in the expansion area) symmetrically distributed on both sides of the transceivers' connection line, the difference in mean absolute errors (MAE) can exceed 3 times.

As a non-intrusive sensing approach, the gait-based human identification technique attracts extensive attention. For the gait-based human identification technique, the unique gait feature is captured and extracted. Owing to the strong environment robustness and good privacy protection, the radar, especially the single-input multiple-output (SIMO) Doppler radar, is proposed as a promising way to capture the gait feature. However, the existing SIMO Doppler radar-based methods require the person to walk along a straight-line trajectory, which hinders their practical application. In this paper, we propose a gait-based human identification system for the weakly constrained trajectory, called CovertEye. In CovertEye, the person can be identified, when he/she walks along variable directions. To this end, we propose a trajectory segmentation algorithm to divide the trajectory into many straight-line trajectory segments. Based on the trajectory segments, we design the gait-based human identification method. In particular, we propose a normalization method to eliminate the differences in the direction of movement and the length among trajectory segments. The normalized signal spectrogram is exploited for the deep learning based feature extraction and human identification. We develop a prototype of the CovertEye system. The extensive experimental results demonstrate that our proposed system can achieve the identification accuracy of 82:4%.

Multi-Camera Active Object Tracking is an attractive technique in the area of intelligent surveillance, where cameras share their observations via the wireless communication to collaboratively track the target. Due to the variability in wireless channel, the dynamic transmission delay between cameras significantly affects the collaboration performance, especially when the tracking is time-sensitive. In this paper, we propose a channel-aware multi-camera active object tracking (CAMAOT) system, to achieve the stable and improved tracking performance. Specifically, a communication decision module is designed in CAMAOT, where the cameras’ communication graph and communication resource allocation adapt to the channels. Our experiments demonstrate that for time-varying channels, CAMAOT has a stable performance improvement over other systems, particularly when the communication resources are limited.

Multi-agent systems (MASs) have emerged as effective means to accomplish important tasks without human involvement in various real-world environments. In MASs, task completion efficiency is determined by the level of cooperation among agents. Meanwhile, achieving high levels of cooperation relies on accurate and comprehensive environmental perception. To this end, agents exchange their local perceptions to expand the scope of their sensing information. However, it limits the improvement of sensing performance by relying solely on information exchange, particularly for mobile target sensing. To address this, we introduce the integrated sensing and communication (ISAC) technique to MASs. This enables the agents to perform distributed radio sensing, while concurrently exchanging their local perceptions. In this article, we propose an ISAC-based MAS framework, where agents can dynamically determine ISAC strategies and cooperatively perceive the environment through ISAC operations. The features of the proposed framework are elucidated and compared with existing networked ISAC systems and communication-centric MASs. For the proposed framework, we suggest a deep reinforcement learning (DRL)-based system design. Simulation results demonstrate the effectiveness of the proposed framework. Finally, we discuss potential challenges and opportunities for future research.

The model-based Wi-Fi sensing approach has shown advantages on facilitating the development of more robust sensing systems, due to its capability of revealing the physical and mathematical sensing mechanisms without the requirement of a large set of training data. Existing models usually treat the sensing target as a particle and characterize the propagation of Wi-Fi signals accordingly, i.e., the effect of target size is ignored. However, in most real-world scenarios, the sensing targets are non-particle and different targets usually have different sizes. Considering that the size difference may have a significant impact on the sensing performance, it is necessary to develop a size-aware model to enrich the Wi-Fi sensing theory. To fill this gap, we propose a non-particle target oriented Wi-Fi sensing model, aiming to describe the relationship between the Wi-Fi signal and the target size. Specifically, by extending the classical particle target oriented Wi-Fi Fresnel zone model, we characterize and quantify the reflected signals from different parts of the non-particle target in a more fine-grained manner. We find the amplitude of the reflected Wi-Fi signals increases and decreases periodically along with the changing of the target size, which is called as the oscillation phenomenon. To validate the proposed model, we implement two sensing applications with a pair of transceivers, including a respiration detection system and a target size measurement system. Extensive experiments demonstrate the correctness and the usefulness of the proposed size-aware model. To the best of our knowledge, this is the first theoretical model that reveals the effect of target size on Wi-Fi sensing.

Predictability is an emerging metric that quantifies the highest possible prediction accuracy for a given time series, being widely utilized in assessing known prediction algorithms and characterizing intrinsic regularities in human behaviors. Lately, increasing criticisms aim at the inaccuracy of the estimated predictability, caused by the original entropy-based method. In this brief report, we strictly prove that the time series predictability is equivalent to a seemingly unrelated metric called Bayes error rate that explores the lowest error rate unavoidable in classification. This proof bridges two independently developed fields, and thus each can immediately benefit from the other. For example, based on three theoretical models with known and controllable upper bounds of prediction accuracy, we show that the estimation based on Bayes error rate can largely solve the inaccuracy problem of predictability.

Visual Simultaneous Localization and Mapping (Visual SLAM) is a methodology that empowers a robot to construct a map of an unfamiliar environment and determine its own position within the map solely based on visual data. However, Visual SLAM faces many challenges in indoor scenarios, such as low lighting, dynamic objects, textureless surfaces, and motion blur. To overcome these difficulties, we propose an indoor Visual SLAM system that combines ORB_SLAM3 with YOLOv8 1 to achieve high accuracy while maintaining robust performance in dynamic environments. Besides, the proposed system also uses the Inertial Measurement Unit (IMU) to aid the SLAM process. The IMU provides complementary information to the camera, such as orientation, acceleration, and angular velocity, which can improve the robustness and accuracy of the Visual SLAM system. Our system undergoes evaluation on two public datasets, as well as in real-world scenarios. To assess its performance, we compare it against alternative Dynamic SLAM approaches. The outcomes demonstrate that our system effectively handles intricate dynamic scenes that involve multiple moving objects, yielding state-of-the-art (SOTA) results. We have released our code on: https://github.com/SimonWXW/ORB-YOLO.