Robust speech recognition systems rely on cloud service providers for inference. It needs to ensure that an untrustworthy provider cannot deduce the sensitive content in speech. Sanitization can be done on speech content keeping in mind that it has to avoid compromising transcription accuracy. Realizing the under-utilized capabilities of tiny speech foundation models (FMs), for the first time, we propose a novel use: enhancing speech privacy on resource-constrained devices. We introduce SpeechShield, an edge/cloud privacy preserving speech inference engine that can filter sensitive entities without compromising transcript accuracy. We utilize a timestamp based on-device masking approach that utilizes a token to entity prediction model to filter sensitive entities. Our choice of mask strategically conceals parts of the input and hides sensitive data. The masked input is sent to a trusted cloud service or to a local hub to generate the masked output. The effectiveness of SpeechShield hinges on how well the entity time segments are masked. Our recovery is a confidence score based approach that chooses the best prediction between the cloud and the on-device model. We implement SpeechShield on a 64 bit Raspberry Pi 4B. Experiments show that our solution leads to robust speech recognition without forsaking privacy. SpeechShield with < 100 MB memory, achieves state-of-the-art (SOTA) speech transcription performance while filtering about 83% of private entities directly on-device. SpeechShield is 16x smaller in memory, 3.3x faster and 17x more compute efficient than prior privacy preserving speech frameworks and has a relative reduction in word error rate (WER) by 38.8-77.5% when compared to existing offline transcription services.
Modern smart glasses, when combined with Large Language Models (LLMs), offer a promising new paradigm for assisting visually impaired individuals in daily navigation and scene understanding. Yet, the effectiveness of such assistants depends critically on factors such as camera Angles of View (AoV), semantic extraction, network conditions, and model selection, which have not been systematically studied. To fill this gap, we construct a dataset of egocentric video sequences with multiple AoVs and systematically generated Q&A, and we implement an LLM-based assistant with edge offloading to evaluate different design choices. In particular, we considered four representative multimodal LLMs: MiniCPM-o 2.6 8B, LLaVA-OneVision 7B, Qwen2.5-VL 7B, and Qwen2.5-VL 3B, to cover a diverse range of model architectures and sizes. Through extensive experiments, we find that: (i) current LLMs are still limited in recognizing 360 degrees videos, but semantic extractors improve accuracy by up to 33.77% with minimal impact on delay; (ii) capturing with 360 degrees cameras raises accuracy by an average of 20.6% and achieves up to 85.64% on position-sensitive queries without additional inference time; (iii) higher-bandwidth networks such as WiFi reduce transmission delay, resulting in acceptable response time (sub 1-second); and (iv) different LLMs perform better on different question categories, highlighting the need for careful modeling and offloading strategies in future work.
We introduce a drone-centric workflow that complements fixed trail cameras to deliver objective, standardized, and scalable assessments of park and trail infrastructure. Traditional human audits, such as PARA and the Boston Block Walk, are labor intensive and subjective and lack the corridor scale perspective needed for comprehensive maintenance planning. Our system addresses this gap by using (i) a multirotor UAV equipped with synchronized thermal and RGB cameras to document surface conditions over vast areas and quantify defect dimensions, and (ii) low-cost fixed cameras to continuously monitor user counts, activity types, and intensity (MET) values. Drone imagery is calibrated via homography to convert pixel measurements to real-world units, and a thermal RGB data fusion pipeline detects cracks, moisture-softened patches, and other defects. A case study on a community tennis court demonstrated a mean absolute percentage error (MAPE) of 11.6% between drone-estimated and tape-measured crack lengths. Along Delaware's Jack A. Markell and James F. Hall trails, camera-based usage analytics revealed patterns consistent with intercept surveys: approximate to 20% of the users were cyclists and approximate to 98% engaged in recreational activities, while estimating speed-derived METs for walkers, runners, and cyclists. Compared to an 11-mile manual audit that required 26 raters, our approach reduced costs by approximately $4,000 and produced reusable digital evidence. We conclude that drones are the necessary backbone for corridor-scale, time-bounded condition assessments, while fixed cameras remain essential for long-term use monitoring; together, they enable auditable, repeatable, and cost-effective management of urban greenspace.
We showcase the workflow of PAVE (Pedestrian Awareness Via Edge analytics), a scalable system for real-time video analytics that leverages street cameras to improve pedestrians' safety while maintaining their privacy. PAVE distributes computation across edge servers and end-user mobile devices. Cameras' live streams are processed at the edge to forecast vehicles' trajectories and detect danger zones. Pedestrians' mobile devices then locally determine if the user is inside a danger zone and trigger timely alerts via a custom iOS app. In addition, anonymized metadata, such as pedestrian and vehicle positions, speeds, and directions, are aggregated and displayed on a public map for broader situational awareness. We evaluated PAVE's performance through implementation on the NSF COSMOS testbed's edge server while processing real-time video stream from cameras in diverse urban environments. Live field tests at an intersection in New York City show that PAVE can alert at-risk pedestrians about 0.9 s before a vehicle reaches them. With low-latency cameras, this lead time extends to around 1.6 s which is within the 1-2 s window pedestrians typically need to react.
The edge is rapidly emerging as a critical environment for AI, vision, robotics, and industrial workloads, where devices are often heterogeneous and resource-constrained. Traditional container-first stacks introduce overhead that reduces available compute for latency-sensitive applications. To address this, we present the Edge Microvisor Toolkit (EMT) and the companion OS Image Composer, which together form a lightweight, customizable substrate for edge computing.
This poster paper examines the operating challenges that prevent far-edge AI systems from scaling, using Kubernetes-based agricultural drone deployments as a case study. Far-edge AI systems perform complex inference on data collected directly from the physical world, transforming fields from ecology to agronomy. However, these systems face diverse operating contexts including strict network firewalls, intermittent outages, and fully offline processing. We introduce inflection points, moments where edge site heterogeneity unexpectedly trigger failures, increase latency, and degrade AI inference. We propose research on inflection points that would allow container platforms to sense local operating context and adapt at runtime to ensure reliable execution at far-edge sites.
EdgeAI systems are increasingly employing computer vision applications to enable intelligent, on-device decision-making in real-time. However, these deployments face highly dynamic operational conditions, with fluctuating constraints on latency, power availability, and memory resources. Deep Neural Networks (DNN), which follow fixed computational execution flows, lack the flexibility to adapt to such variability, resulting in inefficient and suboptimal performance in edge scenarios. This underscores the need for architectures that are not only efficient but also dynamically scalable at runtime. In this paper, we propose Elastoformer: A framework that transforms conventional neural networks (NN) into Elastic NN capable of real-time elastic inference. Unlike the conventional bag-of-models approach, which requires maintaining multiple independent models for different operating conditions, Elastoformer offers a single, modular solution that dynamically switches between multiple modes of operation at runtime, adapting efficiently to the changing computational budgets of edge devices without the overhead of managing separate models. Experiments reveal that our framework achieves up to 85% reduction in computation FLOPs, 50% reduction in latency and 76% reduction in memory overhead, while showcasing the architecture agnostic nature of the framework across both Vision Transformers and CNNs. Our code is available at https://github.com/sudaksh14/Elastoformer.
Dilemma zone (DZ) protection is vital for intersection safety, yet legacy detectors often lack the spatial resolution and responsiveness needed to track fast-changing vehicle dynamics. This paper presents DV-EISOS, an edge-deployed, vision-driven system for real-time DZ detection and adaptive signal control. Running on Jetson AGX Orin, DV-EISOS integrates YOLOv8 for vehicle detection, ByteTrack for multi-object tracking, homography-based mapping, and time-to-intersection logic to convert RTSP camera feeds into actionable kinematic metrics with millisecond-level latency. Through an NTCIP/SNMP interface, the system issues yellow-phase extensions only when approaching vehicles are assessed as high risk. A field deployment at a signalized intersection in Bellevue, WA demonstrated accurate speed estimation (RMSE < 2 mph), timely phase adaptation, and responses to similar to 10% of potential DZ events. These results indicate that edge AI can deliver scalable, low-latency safety control using existing cameras in real-world urban environments.
Open-vocabulary object detection (OVD) is crucial for handling dynamic real-world driving scenarios. Inspired by YOLO-World, we propose OpenVocab-Auto, an open-vocabulary object detection framework with driving-aware multi-scale feature fusion for autonomous driving scenarios. Our system introduces three key innovations: (1) a context-adaptive prompt engine that significantly reduces computational overhead compared to global prompt strategies, (2) hierarchical vision-language alignment for improved small object detection, and (3) real-time optimization achieving 27 FPS on NVIDIA Jetson AGX Orin through TensorRT acceleration. On RTX 3080 (FP16 full model), the framework achieves 0.923 F1 for parking space detection and 0.847 F1 for zero-shot obstacle recognition. On Jetson Orin (TensorRT INT8 model), the corresponding scores are 0.811 and 0.333, respectively, under the same evaluation protocol.
The advancement of machine learning systems depends on large-scale, high-quality datasets. However, corpora drawn from user-generated and proprietary domains are often laden with sensitive information, posing significant privacy, security, and compliance risks. Conventional anonymization methods, which focus on removing explicit identifiers, can degrade downstream performance and leave the more nuanced challenge of implicit privacy leakage unresolved. This form of leakage allows for sensitive attributes such as author identity, demographics, or personality to be inferred from writing style alone. To address this, we present a privacy-preserving text rewriting framework based on guided reinforcement learning. Our approach features a composite reward function that operates over disentangled semantic and stylistic representations to preserve utility while enforcing style convergence and author anonymity. Empirical validation demonstrates substantial improvements on implicit privacy metrics without sacrificing semantic fidelity, yielding a scalable, model-agnostic solution for privacy-preserving data generation in the age of Large Language Models.
In federated learning for multimodal models, balancing global generalization and local personalization remains challenging due to heterogeneous client data distributions. To address this, we propose a novel federated learning framework employing dual Low-Rank Adaptation (LoRA) adapters within a frozen Contrastive Language-Image Pre-training (CLIP) backbone. Each client maintains both global and local adapters, dynamically orchestrated via a lightweight gating network that adaptively fuses the adapters based on input-specific features. Unlike traditional parameter averaging, our server aggregates client knowledge through federated distillation on a compact reference dataset, effectively mitigating parameter conflicts across clients. Experiments demonstrate that our adaptive fusion strategy significantly improves personalized representation quality, outperforming standard LoRA-based federated approaches, especially in scenarios with diverse local data distributions.
Despite advances in hardware acceleration, implementing AI on mobile devices is difficult when tight real-time latency bounds have to be met without compromising accuracy. A simple solution is edge offload: using a low-latency wireless network to perform the AI on a nearby cloudlet. This approach also avoids the software engineering effort of downsizing cloud-based AI. In this paper, we experimentally compare on-device and offloaded AI execution by introducing a set of new benchmarks for computer vision tasks. The results show that edge offload is Pareto-optimal across accuracy and latency. It also greatly reduces on-device energy usage.
Multi-view camera-based 3D object detection through unified Bird's Eye View (BEV) representation has become popular for autonomous driving due to its low cost, but efficiently inferring precise spatial and temporal information from cameras alone remains a significant challenge. Transformer-based approaches have shown substantial performance improvements but have the drawback of quadratic memory complexity - making these architectures ill-suited for edge deployment. Recently, State Space Models (SSMs) offer a more favorable balance of computational efficiency and performance in 2D vision, suggesting that they could help here as well. We present TinyBEV, an efficient BEV framework for multi-view 3D perception. For spatial modeling, we replace cross attention with SSMs that fusing BEV and camera images with linear complexity. For temporal modeling, we adopt a lightweight, linear-complexity history-fusion scheme that uses explicit time conditioning and channel-level aggregation instead of cross-frame attention. Both fusion strategies follow small constant scaling with respect to history length and enabling edge-friendly deployment. Experiments on NuScenes datasets demonstrate that TinyBEV is comparable with other state-of-the-art methods across diverse visual perception metrics with advantages in computational efficiency.
The flight time of uncrewed autonomous vehicles (UAVs) is constrained by its battery capacity, restricting its application in long-duration missions. To address this challenge, we propose eFlight, a hybrid scheme that uses a reinforcement-learning heuristic to augment A* for path finding. eFlight reduces both node expansions and computation time while finding energy-efficient paths in obstacle-dense 3D airspace. We compare eFlight with conventional path-planning algorithms for point-to-point flights on areas of various dimensions and with various obstacle densities. The results show that eFlight achieves a dual advantage: finding low-energy paths with short computation times. In high-density obstacle environment, eFlight identifies the lowest energy consumption path in 89.5% of the trials. Compared to the baseline scheme, eFlight reduces computation time by 90.6% +/- 26.6% and energy by 7.13% +/- 9.96%.
Serverless and Edge Computing ought to be a perfect match to create flexible, efficient, and responsive applications for emerging areas like augmented reality and autonomous vehicles. Unfortunately, current serverless designs incur high overheads-preventing submillisecond execution-and consume large amounts of resources-preventing dense deployment within constrained edge environments. We present a platform to overcome these challenges, while providing strong isolation and performance in multi-tenant edge data centers. Our system, SledgeScale achieves this through lifecycle management of lightweight WebAssembly sandboxes that can be rapidly instantiated for each request, a high performance kernel bypass-based communication framework, and load- and deadline-aware request dispatch and scheduling algorithms. Our evaluation shows that we can reach 1M req/second using 15 cores. Under a log normal distribution workload, we can sustain 6x and 48x more load than DARC and Shinjuku scheduling algorithms, respectively, for a target 99.9th percentile slowdown of 200x. We support unprecedented levels of function density-maintaining 99th percentile latency under 60 microseconds and throughput over 250K req/sec for 10,000 distinct functions running on six cores.
Mobile Augmented Reality (MAR) must balance AI latency and visual quality of virtual objects amid user and scene dynamics. We present an edge-assisted RL framework that jointly optimizes AI task allocation across CPU, GPU, and NPU and the triangle ratio of rendered virtual objects. A lightweight on-device context predictor forecasts near-future states, while a multi-actor, single-reward model on the edge makes decisions via a one-step contextual bandit with UCB-based actor selection. This design enables proactive, context-aware adaptation and fast policy reuse. Experimental tests on real smartphones and edge servers show that our approach reduces exploration by up to 57.1%, extends exploitation up to 12x, and improves reward by 193% over state-of-the-art baselines.
