Training binary neural networks (BNNs) from scratch is dominated by the straight-through estimator (STE), whose forward/backward mismatch produces severe accuracy degradation as networks deepen. We study an orthogonal axis: when and where binarization is enforced during training. We introduce StoMPP (Stochastic Masked Partial Progressive Binarization), which gradually replaces clipped weights and activations with their hard binary counterparts layer by layer from input to output, using stochastic partial masks with soft refresh. StoMPP delivers two complementary benefits. As a standalone training rule, it provides a fully STE-free procedure that improves over vanilla STE with gains that grow with depth (ResNet-50 BNN: +18.0/+13.5/+3.8 on CIFAR-10/100/ImageNet), and the pattern holds across ResNet-18/34/50, MobileNetV2, and BERT fine-tuning. Composed with surrogate gradients by applying STE only to frozen entries, it reaches +27.1/+19.8/+17.7 over vanilla STE on the same setting. Underlying both regimes is a single mechanistic finding: progression order is decisive. Forward layerwise progression prevents depth collapse, reverse progression collapses to near-chance, and binary-weight networks (without binary activations) are insensitive to order. We trace this asymmetry to activation-induced gradient blockades: a committed binary activation severs gradient flow upstream, and ordering controls when these blockades form. To isolate the progression's contribution from any benefit conferred by STE, we conduct all ablations in the STE-free regime; the resulting characterization (schedule, refresh, ordering, dynamics) thus reflects the progression itself rather than its interaction with surrogate gradients.

Spatial reasoning is fundamental to auditory perception, yet current audio large language models (ALLMs) largely rely on unstructured binaural cues and single- step inference. This limits both perceptual accuracy in direction and distance estimation and the capacity for interpretable reasoning. Recent work such as BAT demonstrates spatial QA with binaural audio, but its reliance on coarse categorical labels (left, right, up, down) and the absence of explicit geometric supervision constrain resolution and robustness. We introduce the $\textbf{Spatial-Acoustic Geometry Encoder (SAGE}$), a geometry-aware audio encoder that aligns binaural acoustic features with 3D spatial structure using panoramic depth images and room-impulse responses at training time, while requiring only audio at inference. Building on this representation, we present $\textbf{OWL}$, an ALLM that integrates $\textbf{SAGE}$ with a spatially grounded chain-of-thought to rationalize over direction-of-arrivals (DoA) and distance estimates. Through curriculum learning from perceptual QA to multi-step reasoning, $\textbf{OWL}$ supports o’clock-level azimuth and DoA estimation. To enable large-scale training and evaluation, we construct and release $\textbf{BiDepth}$, a dataset of over one million QA pairs combining binaural audio with panoramic depth images and room impulse responses across both in-room and out-of-room scenarios. Across two benchmark datasets, our new $\textbf{BiDepth}$ and the public SpatialSoundQA, $\textbf{OWL}$ reduces mean DoA error by $\textbf{11$^{\circ}$}$ through $\textbf{SAGE}$ and improves spatial reasoning QA accuracy by up to $\textbf{25}$% over BAT. Our dataset and code are available at: https://anonymous.4open.science/r/OWL-ICLR-26/

Recent advances in model design and self-supervised audio representations have improved speech and audio understanding, yet infant-centered naturalistic recordings remain challenging due to limited labeled data, low signal-to-noise ratio, and cross-family domain shifts. We present a family-conditioned, multi-tier audio tagger that combines a LoRA-finetuned Whisper encoder with a lightweight, target-speaker-aware Transformer for long-context inference and framewise prediction across tiers. To improve temporal coherence, we incorporate a simple sequence-level smoothing loss, and to enhance robustness across households, we introduce a factorized speaker-token design with a shared tier token and a learned family-specific offset, reducing family bias and promoting generalizable representations. Together, these choices enable efficient and effective infant-centered audio tagging of daylong audio recordings in home environments.

Head-worn devices such as augmented-reality (AR) and smart glasses introduce a previously overlooked form of audio degradation: hair noise, caused by the wearer’s hair brushing against device frames and embedded microphones. To the best of our knowledge, this phenomenon has not been systematically studied. This paper addresses this gap through three contributions. First, we conduct a user study quantifying the perceptual annoyance of hair noise. Second, we introduce the Hair Noise Mitigation (HNM) dataset, the first multi-channel corpus of hair noise collected across diverse real-world conditions. We further characterize its spectral and spatial properties, revealing a non-stationary and directionally dependent nature. Finally, we propose online and offline semi-supervised nonnegative matrix factorization (NMF) methods as benchmark mitigation approaches, showing perceptual gains that motivate further research. Together, these contributions establish hair noise as a distinct challenge for wearable audio systems and lay the groundwork for tailored enhancement techniques. The HNM dataset can be accessed at https://subrata132.github.io/hnm-project/

We investigate progressive freezing as an alternative to straight-through estimators (STE) for training binary networks from scratch. Under controlled training conditions, we find that while global progressive freezing works for binary-weight networks, it fails for full binary neural networks due to activation-induced gradient blockades. We introduce StoMPP (Stochastic Masked Partial Progressive Binarization), which uses layerwise stochastic masking to progressively replace differentiable clipped weights/activations with hard binary step functions, while only backpropagating through the unfrozen (clipped) subset (i.e., no straight-through estimator). Under a matched minimal training recipe, StoMPP improves accuracy over a BinaryConnect-style STE baseline, with gains that increase with depth (e.g., for ResNet-50 BNN: +18.0 on CIFAR-10, +13.5 on CIFAR-100, and +3.8 on ImageNet; for ResNet-18: +3.1, +4.7, and +1.3). For binary-weight networks, StoMPP achieves 91.2% accuracy on CIFAR-10 and 69.5% on CIFAR-100 with ResNet-50. We analyze training dynamics under progressive freezing, revealing non-monotonic convergence and improved depth scaling under binarization constraints.

Procedural tasks such as furniture assembly and home repair impose substantial cognitive demands because users must interpret instructions, track task progress, reason about spatial state, and recover from errors while performing physical actions. Prior multimodal assistants have shown promise for procedural guidance, but most rely on cloud inference and fixed always-on perception, making them poorly suited to privacy-sensitive, latency-critical domestic settings. We present ProcAgent, a fully on-device, agentic, vision-based procedural assistant for real-time adaptive guidances on a single NVIDIA Jetson AGX Orin. ProcAgent uses a propose-and-verify architecture that combines low-latency continuous perception, a symbolic task graph, on-demand vision-language verification, and an LLM-based interaction agent. The system continuously proposes user progress, invokes expensive visual reasoning only when ambiguity or likely deviation arises, and supports both reactive question answering and proactive intervention with human-in-the- loop confirmation. We evaluate ProcAgent along four dimensions: perception accuracy, reasoning, task-level performance, and user experience. Despite running entirely on-device, the system maintains responsive interaction, resolving text-only queries in approximately 2 seconds and visually grounded queries in approximately 8 seconds. In a user study with 10 participants completing assembly tasks, ProcAgent receives positive ratings for comprehensibility, actionability, and privacy comfort. These results show that adaptive procedural assistance can be achieved entirely on edge hardware without sacrificing usability.

ECG is an essential diagnostic tool that offers important insight into a person's cardiac and general health. The rise of intelligent wearable devices has opened a new avenue for clinicians and individuals to capture long term ECG data—albeit with fewer leads than the 12 leads that are typically used clinically, which can be vital for identifying and addressing health concerns. In this work, a multi-task convolutional neural network (CNN) classifier was used to study the influence of various combinations of ECG leads in interpretation of 71 cardiac statements spanning cardiac diagnostics, form, and rhythm. Results of this analysis suggest that the subset of limb leads I and II and chest leads V1, V3, and V6 can be used to identify several cardiac statements without loss of performance (average macro AUC of 0.903) when compared to a model trained using all 12- leads (average macro AUC of 0.905; p = 1). A hybrid CNNLSTM (long short-term memory) model was developed to reconstruct the missing chest leads. The highest performing lead reconstructor achieved an average R2 score of 0.835 when reconstructing three chest leads. This architecture was proposed as the foundation for a wearable system that could record a limited number of ECG leads while also providing a 12-lead ECG for clinical applications.

Existing pruning methods are typically applied during training or compile time and often rely on structured sparsity. While compatible with low-power microcontrollers (MCUs), structured pruning underutilizes the opportunity for fine-grained efficiency on devices without SIMD support or parallel compute. To address these limitations, we introduce UnIT (Unstructured Inference-Time pruning), a lightweight method that dynamically identifies and skips unnecessary multiply-accumulate (MAC) operations during inference, guided by input-specific activation patterns. Unlike structured pruning, UnIT embraces irregular sparsity and does not require retraining or hardware specialization. It transforms pruning decisions into lightweight comparisons, replacing multiplications with threshold checks and approximated divisions. UnIT further optimizes compute by reusing threshold computations across multiple connections and applying layer- and group-specific pruning sensitivity. We present three fast, hardware-friendly division approximations tailored to the capabilities of common embedded platforms. Demonstrated on the MSP430 microcontroller, UnIT achieves 11.02

Wearables generate rich motion data, yet current systems only classify what happened - failing to support natural questions about why it happened or what it means. We introduce LLaSA (Large Language and Sensor Assistant), a compact 13B model that enables ask-anything, open-ended question answering grounded in raw IMU data. LLaSA supports conversational, context-aware reasoning - explaining the causes of sensor-detected behaviors and answering free-form questions in real-world scenarios. It is tuned for scientific accuracy, coherence, and response reliability. To advance this new task of sensor-based QA, we release three large-scale datasets: SensorCaps, OpenSQA, and Tune-OpenSQA. Together, these resources define a new benchmark for sensor-language models. LLaSA consistently produces interpretable, causal answers and outperforms commercial LLMs across both public and real-world settings. Our code repository and datasets can be found at this https URL

Wearable systems can recognize activities from IMU data but often fail to explain their underlying causes or contextual significance. To address this limitation, we introduce two large-scale resources: SensorCap, comprising 35,960 IMU-caption pairs, and OpenSQA, with 199,701 question-answer pairs designed for causal and explanatory reasoning. OpenSQA includes a curated tuning split (Tune-OpenSQA) optimized for scientific accuracy, narrative clarity, and diagnostic insight. Leveraging these datasets, we develop LLaSA (Large Language and Sensor Assistant), a family of compact sensor-aware language models (7B and 13B) that generate interpretable, context-rich responses to open-ended questions grounded in raw IMU data. LLaSA outperforms commercial LLMs, including GPT-3.5 and GPT-4o-mini, on benchmark and real-world tasks, demonstrating the effectiveness of domain supervision and model alignment for sensor reasoning.

Wearable devices have begun to incorporate machine learning models to assist with detection of various cardiac conditions. In this work, we developed a multi-task convolutional neural network to simultaneously predict $\mathbf{7 5}$ diagnostic, form and rhythm statements from 10-s duration, 12-lead ECGs. The model, originally developed off-line in TensorFlow, was converted to the FlatBuffers format for on-edge AI using the LiteRT toolset. Posttraining quantization was used to compare different numerical precisions in terms of model size, model performance and inference time. Classifier performance for the 12-lead configuration was consistent between the 32-bit floating point model (“float32” baseline), the dynamic range quantized model (DR) and the float16 model $(p=0.92)$ with an average macro AUC score of 0.893 with all output statements considered. A large degradation in classification performance was observed for 8-bit integer quantization (int8) which yielded an average macro AUC score of 0.513 for the 12-lead configuration across all statements. To address class imbalance, minority classes were removed. Reducing the number of statements to $\mathbf{4 1}$ classes increased macro F1 score by an average of 72.6% (to a mean value about 0.358) for the float32, float16 and DR quantized models.

We present the first ever dataset of deliberately aliasing audio. Aliasing, an acoustic phenomenon which adds noise and other abnormalities to a recording, occurs when the sample rate of audio is below the Nyquist sampling rate. The Nyquist sampling rate is defined as 2x the highest frequency present in the audio. Sampling below the Nyquist rate folds the upper frequencies into the lower frequencies causing distortion because the lower sample rate cannot accurately capture the high frequencies. Using three standard audio datasets, we curated several sub-datasets of aliasing audio at different sample rates. This dataset is necessary in the age of IoT because always-on microphone devices will benefit from lower sample rates which require less power to sample the audio and less bandwidth to transmit recordings. Lower sample rates will cause aliasing and our dataset is the first dataset pre-made to test the effects of aliasing on downstream tasks like audio classification. The dataset can also be used to build algorithms to classify audio as aliasing or not.

Spoken Language Understanding (SLU) systems must balance performance and efficiency, particularly in resource-constrained environments. Existing methods apply distillation and quantization separately, leading to suboptimal compression as distillation ignores quantization constraints. We propose QUADS, a unified framework that optimizes both through multi-stage training with a pre-trained model, enhancing adaptability to low-bit regimes while maintaining accuracy. QUADS achieves 71.13% accuracy on SLURP and 99.20% on FSC, with only minor degradations of up to 5.56% compared to state-of-the-art models. Additionally, it reduces computational complexity by 60-73x (GMACs) and model size by 83-700x, demonstrating strong robustness under extreme quantization. These results establish QUADS as a highly efficient solution for real-world, resource-constrained SLU applications.

Accurate estimation of resting noise standard deviation (σnoise) in surface electromyography (EMG) is essential for EMG amplitude estimation, particularly during low-level contractions where signal-to-noise ratios are low. Conventional σnoise estimates are made from rest-state EMG recordings, but such data are not always available or routinely recorded in real-world settings. This study compared three methods of σnoise estimation: 1) direct rest-state measurement ("truth"), and two methods intended to remove the need for explicit rest trials: 2) a novel machine learning (ML) approach utilizing active contractions, and 3) a fixed σnoise value of 3 % maximum voluntary EMG (MVE). The ML model was trained on simulated EMG, then fine-tuned on EMG recordings from constant-force and force-varying elbow contraction of 62 subjects spanning three different EMG acquisition systems. Direct resting σnoise measurements had a median absolute inter-trial difference of 0.06 % MVE. ML had a median absolute difference from rest-based σnoise of 1.4 % MVE. The fixed σnoise approach had a median absolute difference (1.71 % MVE) from rest-based σnoise that did not differ significantly from the ML results. In a separate evaluation of baseline EMG noise reduction, omission of noise correction performed worst, ML and the fixed σnoise value performed similarly but statistically better (45 % noise reduction compared to no noise correction), and noise calibration from a rest contraction performed statistically best (75 % noise reduction compared to no noise correction). These results suggest the feasibility of ML-or use of a fixed σnoise value-as alternatives to direct EMG noise measurement, enabling more reliable EMG analysis in scenarios where rest-state data cannot be collected.

Accurate sound source localization (SSL) requires consistent multichannel data for reliable degree of arrival (DoA) estimation. However, intermittently powered batteryless systems often suffer from incomplete sensor data due to the stochastic nature of energy harvesting. Existing methods struggle with missing channels, leading to significant performance degradation. In this paper, we propose $\textit{LOCUS}$, a novel deep learning-based system designed to recover corrupted features for SSL in batteryless systems. $\textit{LOCUS}$ addresses missing data by leveraging information entropy estimation and conditional interpolation, combining three modules: (1) Information-Weighted Focus (InFo), which identifies and quantifies corrupted data elements, (2) Latent Feature Synthesizer (LaFS), which synthesizes missing features, and (3) Guided Replacement (GRep), which intelligently replaces missing elements while preserving valid data. We demonstrate significant performance improvements using two datasets: DCASE and LargeSet, where $\textit{LOCUS}$ achieves up to $36.91\%$ lower DoA error compared to existing methods. Real-world evaluations across three environments with intermittent power sources show a $25.87-59.46\%$ improvement in performance when channels are stochastically missing. Additionally, we release a 50-hour multichannel dataset to support further research in SSL.

Multimodal question answering (QA) often requires identifying which video, audio, or sensor tokens are relevant to the question. Yet modality disagreements are common: off-camera speech, background noise, or motion outside the field of view often mislead fusion models that weight all streams equally. We present RAVEN, a unified QA architecture whose core is QuART, a query-conditioned cross-modal gating module that assigns scalar relevance scores to each token across modalities, enabling the model to amplify informative signals and suppress distractors before fusion. RAVEN is trained through a three-stage pipeline comprising unimodal pretraining, query-aligned fusion, and disagreement-oriented fine-tuning - each stage targeting a distinct challenge in multi-modal reasoning: representation quality, cross-modal relevance, and robustness to modality mismatch. To support training and evaluation, we release AVS-QA, a dataset of 300K synchronized Audio-Video-Sensor streams paired with automatically generated question-answer pairs. Experimental results on seven multi-modal QA benchmarks - including egocentric and exocentric tasks - show that RAVEN achieves up to 14.5% and 8.0% gains in accuracy compared to state-of-the-art multi-modal large language models, respectively. Incorporating sensor data provides an additional 16.4% boost, and the model remains robust under modality corruption, outperforming SOTA baselines by 50.23%. Our code and dataset are available at https://github.com/BASHLab/RAVEN.

Mindfulness training is widely recognized for its benefits in reducing depression, anxiety, and loneliness. With the rise of smartphone-based mindfulness apps, digital meditation has become more accessible, but sustaining long-term user engagement remains a challenge. This paper explores whether respiration biosignal feedback and mindfulness skill estimation enhance system usability and skill development. We develop a smartphone's accelerometer-based respiration tracking algorithm, eliminating the need for additional wearables. Unlike existing methods, our approach accurately captures slow breathing patterns typical of mindfulness meditation. Additionally, we introduce the first quantitative framework to estimate mindfulness skills---concentration, sensory clarity, and equanimity---based on accelerometer-derived respiration data. We develop and test our algorithms on 261 mindfulness sessions in both controlled and real-world settings. A user study comparing an experimental group receiving biosignal feedback with a control group using a standard app shows that respiration feedback enhances system usability. Our respiration tracking model achieves a mean absolute error (MAE) of 1.6 breaths per minute, closely aligning with ground truth data, while our mindfulness skill estimation attains F1 scores of 80-84% in tracking skill progression. By integrating respiration tracking and mindfulness estimation into a commercial app, we demonstrate the potential of smartphone sensors to enhance digital mindfulness training.

Recent advancements in Ultra-Wideband (UWB) radar technology have enabled contactless, non-line-of-sight vital sign monitoring, making it a valuable tool for healthcare. However, UWB radar's ability to capture sensitive physiological data, even through walls, raises significant privacy concerns, particularly in human-robot interactions and autonomous systems that rely on radar for sensing human presence and physiological functions. In this paper, we present Anti-Sensing, a novel defense mechanism designed to prevent unauthorized radar-based sensing. Our approach introduces physically realizable perturbations, such as oscillatory motion from wearable devices, to disrupt radar sensing by mimicking natural cardiac motion, thereby misleading heart rate (HR) estimations. We develop a gradient-based algorithm to optimize the frequency and spatial amplitude of these oscillations for maximal disruption while ensuring physiological plausibility. Through both simulations and real-world experiments with radar data and neural network-based HR sensing models, we demonstrate the effectiveness of Anti-Sensing in significantly degrading model accuracy, offering a practical solution for privacy preservation.

Acoustic sensing is essential in smart infrastructure for occupancy detection, appliance monitoring, and anomaly alerts, where latency and continuous operation are critical. However, fixed high-rate sampling drains power and storage in embedded systems. Prior adaptive sampling approaches often rely on narrow-band speech, analog filters, multi-microphone arrays, or task-specific heuristics, each increasing hardware complexity or reducing generalizability. We present EfficientMic, a digital, aliasing-aware framework for adaptive acoustic sensing using only a single microphone. Without analog components, EfficientMic detects aliasing from Short-time Fourier Transform (STFT) features using a lightweight XGBoost model and dynamically adjusts the sampling rate in real time to preserve fidelity across diverse sounds. Deployed on a Raspberry Pi and evaluated on two environmental sound classification datasets, EfficientMic reduces storage and energy use by over 70% and 50%, respectively, with classification accuracy within 1.2% of full-rate baselines. This task-agnostic, hardware-independent design enables scalable, low-power sensing for smart environments.

The growing demand for matrix multiplication in artificial intelligence must be met with increased tensor computing efficiency and bandwidth improvements. While AI throughput using digital hardware accelerators has advanced, the potential of analog circuits has been largely untapped. In this paper, we introduce a novel Fourier-Accelerated CMOS-based Tensor Engine (FATE) that aims to optimize the high complexity of matrix multiplication with computation and bandwidth efficiencies. First, our design reduces the computational complexity of matrix multiplication from the traditional $\mathrm{O}\left(N^{3}\right)$ to $\mathrm{O}\left(N^{3} / \mathrm{f}\right)$, where $\mathrm{f}$ is a number of frequency carriers up to $N$. Second, our circuit dramatically reduces the bandwidth required for moving vectors by encoding a vector as a summed sine series routed via two physical wires. The test circuit, designed with a 180 nm standard CMOS process, achieves a strong dot product linearity with an $R^{2}$ value of 0.94. This work highlights the untapped potential of analog circuits in modern AI, offering a highly efficient solution to a critical bottleneck in AI computation.