Edge services increasingly use federated learning to personalize on-device models while keeping sensitive data local. In practice, deployments must handle heterogeneity in both client resources and local data distributions. Model-heterogeneous federated learning lowers client cost by allowing each client to train a subnet of a shared supernet, but most subnet-allocation policies are driven by device constraints and do not explicitly account for statistical heterogeneity. This paper proposes Heterogeneity-Aware Subnet Allocation (HASA), a train-only rule that assigns subnet widths based on client heterogeneity scores computed from local training data while enforcing a fixed size-weighted compute budget. This design enables budget-matched comparisons with alternative allocation policies. On an article-title next-word prediction benchmark with seven clients, HASA improves unweighted mean client test accuracy over uniform allocation across 10 matched seeds, increasing mean client test accuracy from 13.82 percent to 14.32 percent, and improves worst-client accuracy on average. In a matched-budget comparison with representative partial-training baselines, HASA achieves the strongest worst-client and tail-client accuracy on this benchmark. A directionality ablation shows that assigning smaller subnets to more heterogeneous clients degrades both mean and tail performance. A cross-domain image-classification study further shows that the effectiveness of heterogeneity-aware allocation depends on how well the heterogeneity score reflects clients' need for additional model width.
Robust speech recognition systems rely on cloud service providers for inference. It needs to ensure that an untrustworthy provider cannot deduce the sensitive content in speech. Sanitization can be done on speech content keeping in mind that it has to avoid compromising transcription accuracy. Realizing the under-utilized capabilities of tiny speech foundation models (FMs), for the first time, we propose a novel use: enhancing speech privacy on resource-constrained devices. We introduce SpeechShield, an edge/cloud privacy preserving speech inference engine that can filter sensitive entities without compromising transcript accuracy. We utilize a timestamp based on-device masking approach that utilizes a token to entity prediction model to filter sensitive entities. Our choice of mask strategically conceals parts of the input and hides sensitive data. The masked input is sent to a trusted cloud service or to a local hub to generate the masked output. The effectiveness of SpeechShield hinges on how well the entity time segments are masked. Our recovery is a confidence score based approach that chooses the best prediction between the cloud and the on-device model. We implement SpeechShield on a 64 bit Raspberry Pi 4B. Experiments show that our solution leads to robust speech recognition without forsaking privacy. SpeechShield with < 100 MB memory, achieves state-of-the-art (SOTA) speech transcription performance while filtering about 83% of private entities directly on-device. SpeechShield is 16x smaller in memory, 3.3x faster and 17x more compute efficient than prior privacy preserving speech frameworks and has a relative reduction in word error rate (WER) by 38.8-77.5% when compared to existing offline transcription services.
Modern smart glasses, when combined with Large Language Models (LLMs), offer a promising new paradigm for assisting visually impaired individuals in daily navigation and scene understanding. Yet, the effectiveness of such assistants depends critically on factors such as camera Angles of View (AoV), semantic extraction, network conditions, and model selection, which have not been systematically studied. To fill this gap, we construct a dataset of egocentric video sequences with multiple AoVs and systematically generated Q&A, and we implement an LLM-based assistant with edge offloading to evaluate different design choices. In particular, we considered four representative multimodal LLMs: MiniCPM-o 2.6 8B, LLaVA-OneVision 7B, Qwen2.5-VL 7B, and Qwen2.5-VL 3B, to cover a diverse range of model architectures and sizes. Through extensive experiments, we find that: (i) current LLMs are still limited in recognizing 360 degrees videos, but semantic extractors improve accuracy by up to 33.77% with minimal impact on delay; (ii) capturing with 360 degrees cameras raises accuracy by an average of 20.6% and achieves up to 85.64% on position-sensitive queries without additional inference time; (iii) higher-bandwidth networks such as WiFi reduce transmission delay, resulting in acceptable response time (sub 1-second); and (iv) different LLMs perform better on different question categories, highlighting the need for careful modeling and offloading strategies in future work.
