Internet of things (IoT) enables intelligent interaction and data sharing, facilitating connectivity among objects and humans. It has been widely applied in smart homes, intelligent transportation, industrial automation, etc. Traditional IoT development processes rely on manual hardware-software collaborative development, which involves a high technical barrier. Low-code development technology significantly lowers this barrier by providing graphical interfaces and highly abstracted programming application programming interfaces. Nevertheless, it still faces limitations in customization capabilities. With the maturity of artificial intelligence (AI) and related technologies represented by large language models, an emerging AI-native paradigm for IoT computing task generation characterized by leveraging AI models to provide high-level semantic representations of IoT application development workflows is gaining traction and presenting novel opportunities for IoT application development. Therefore, on the basis of reviewing the development of software development technology for the IoT, an AI-native framework for generating computing tasks of IoT was proposed in this paper, which was divided into two stages: intent understanding and task planning. Based on this, the key technologies and challenges were systematically analyzed, a comprehensive review of the latest research was provided, and the future development directions of AI-native computing task generation were finally looked forward to.

Video question answering (VideoQA), whose goal is to produce answers through the integration of linguistic and visual understanding, has emerged as a significant research focus. Although Large Multimodal Models (LMMs) and autonomous agent methods have achieved notable advances in VideoQA, excessive computational overhead and restricted multimodal interaction capabilities limit their ability to facilitate the continuous evolution of the VideoQA system. To address the challenge, we introduce DigimonGPT, an evolvable VideoQA agent inspired by cognitive psychology. Specifically, DigimonGPT integrates a multimodal memory mechanism to achieve the continuous evolution of VideoQA systems. An intra-video declarative memory contains fundamental features of the video and semantic contexts extracted from historical QA pairs. Another inter-task procedural memory encodes task-solving experience for further question answering. Additionally, we introduce a hierarchical memory replay mechanism for VideoQA that selects appropriate memories by their relevance and question complexity. Extensive experiments demonstrate that DigimonGPT's accuracy averagely outperforms 13.71% on NExT-QA datasets and 9.89% on Intent-QA datasets over LMM and autonomous agents.

Large language models (LLMs) are widely used in various fields, and the demand for LLM inference on mobile devices is also growing. LoRA technology, due to its high efficiency and low cost, is a basic method for fine-tuning pre-trained models for downstream tasks. However, due to the limited hardware resources of mobile devices, ensuring efficient inference while fully utilizing mobile hardware resources becomes a key issue. Considering the characteristics of heterogeneous computing resources and unified memory architecture on mobile devices, we propose MHP-SERVE, an LLM inference framework for mobile devices. MHP-SERVE ensures effective partitioning of LoRA-based models, guaranteeing that LoRA inference computations run on the CPU and base model computations run on the GPU. It also designs a heterogeneous parallel execution scheme to effectively ensure the correct sequential execution of synchronization points. By using shared memory in a unified memory architecture to share tensors, it optimizes the delay caused by memory copies in existing solutions, achieving efficient CPU-GPU data transfer. Extensive experiments show that this framework reduces the latency of generating the first token by 10

Video object detection stands as a pivotal element within the burgeoning landscape of autonomous driving systems. The exigency to fulfill stringent real-time requisites, while upholding both precision and efficiency in detection, underscores its significance. Although extant methodologies enhance either accuracy or efficiency through the exploitation of spatio-temporal inter-dependencies within the video context, their propensity to conduct detection on discrete frames begets superfluous computations and curbed real-time efficacy. This paper introduces a pioneering approach, called Real-Batch, tailored explicitly to redress this quandary. Real-Batch ingeniously processes batches of video frames uniformly, effectually winnowing out repetitive object detection occurrences. Our methodology is rigorously evaluated on the real-word datasets, scrutinizing four key metrics: accuracy, efficiency, informational value, and adherence to timing constraints. The comprehensive findings substantiate that Real-Batch yields an unparalleled maximal surge in accuracy and efficiency, increasing of 4.2%-13.2% and 24.7%-43.9%, respectively, offering promising advancements for autonomous driving systems.

Millimeter-wave radar has shown significant potential in privacy-preserving human activity recognition. However, the lack of diverse radar datasets across various scenarios poses a challenge to the robustness and generalization of deep learning models. To address this limitation, existing works mainly focus on synthesizing micro-doppler data from video, range-doppler data, which provides an extra dimension, has been overlooked due to challenges caused by signal offsets. In this paper, we propose a comprehensive approach for synthesizing range-doppler data from videos by leveraging computer vision techniques and principles of camera imaging. Furthermore, we implement a map enhancement and classification model to facilitate human activity recognition. Our approach is validated on a custom dataset, where the proposed range-doppler synthesis method and classification model achieve an accuracy of 97.3% for activity recognition tasks. This performance is comparable to that of vision-based HAR methods, demonstrating the effectiveness of our proposed scheme in achieving privacy-preserving human activity recognition.

Embodied AI models now span vision-language-action (VLA) models and world-action models (WAMs), but practical deployment remains fragmented across model-specific Python stacks, backend assumptions, and robot-side glue code, especially on heterogeneous edge devices. Existing inference runtimes are designed mainly for request-response serving and therefore do not satisfy the runtime contract of embodied deployment: multi-rate execution inside closed-loop control, latency-first batch-1 inference on heterogeneous hardware, and extensible embodied interfaces beyond fixed token I/O. We present Embodied.cpp, a portable C++ inference runtime for embodied models. Based on an architectural analysis of representative VLA models and WAMs, Embodied.cpp captures a shared execution path and organizes it into five layers: input adapters, sequence builders, backbone execution, head plugins, and deployment adapters. The runtime provides modular multi-rate execution, latency-first fused inference, and extensible operator and I/O support, enabling deployment across heterogeneous devices, robots, and simulators through one backend abstraction. We evaluate Embodied.cpp on three VLA and two WAM models, using normalized comparisons across Python and C++ quantization configurations. Overall, Embodied.cpp achieves 1.05x-2.70x inference speedups and 7%-77% lower VRAM relative to Python baselines, while maintaining near-baseline success for most configurations. These results show that Embodied.cpp improves deployment efficiency while preserving high control quality across diverse embodied model architectures. Project Link: https://github.com/SEU-PAISys/Embodied.cpp

Video analytics services utilizing multiple deep neural network models (DNNs) are increasingly being adopted in various edge intelligence applications. To avoid the accuracy reduction caused by data drift, existing works leverage continual learning that directly retrains DNN models in multi-model applications. According to our preliminary experiments, we observe that existing continual learning frameworks neglect the accuracy correlation among multiple DNN models and the opportunity to harvest idle resources during model retraining, leading to considerable resource inefficiency. In this paper, we propose FaaSLearner, an algorithm-system co-designed continual learning framework for multi-model video analytics that leverages the agile and fine-grained resource management of serverless computing. Specifically, FaaSLearner proposes the correlation-aware retraining planning to analyze the multi-model correlation, and then selectively retrain DNN models with the greatest in accuracy gain. In addition, FaaSLearner proposes the resource-efficient retraining scheduling to accurately trigger the continual learning, and then harvest the keep-alive periods for retraining tasks without interfering with normal inference. We evaluate FaaSLearner with four common edge video analytics applications with the Azure public dataset. Extensive experiments show that, FaaSLearner improves the average application accuracy up to 31.9% over representative baselines, and harvests more than 2.51× idle memory resources for retraining.

Along with the increasing popularity of Heterogeneous Federated Learning (HFL), existing frameworks are capable of training diverse sub-models on devices with varying resources. However, due to the inherent complexity of balancing system-level trade-offs, these frameworks suffer from a critical usability and configuration problem. Although various HFL methods have been investigated to handle device heterogeneity, there is still a lack of i) an automated mechanism to translate high-level user requirements into optimal configurations, ii) a systematic approach to navigating the accuracy-latency trade-off without expert intervention, and iii) a user-centric paradigm that adapts to specific deployment goals. To address the above issues, this paper introduces LLM-Flex, a novel framework that integrates a Large Language Model (LLM) agent to automate the configuration of HFL systems. By leveraging an LLM to parse natural language intent, LLM-Flex can dynamically select the optimal model architecture and heterogeneity distribution to best fit the user’s goals. Meanwhile, our proposed framework utilizes the robust FlexFL algorithm as its execution engine, inheriting its capabilities for adaptive local pruning and self-knowledge distillation. Comprehensive experimental results show that, compared to state-of-the-art baselines with fixed configurations, LLM-Flex can significantly reduce training latency by up to 24.3% in speed-focused scenarios and improve accuracy by up to 37.3% in performance-focused scenarios, successfully aligning system performance with user intent.

With the surging demand for real-time perception-decision-control closed-loop in distributed embodied intelligent scenarios,traditional cloud-native architectures face severe challenges in concurrent processing and deterministic latency.This paper proposes a 50G-PON end-edge-cloud collaborative architecture for ubiquitous access of intelligent agents.By implementing hierarchical computing resource allocation and deterministic transmission mechanisms,it addresses latency jitter and bandwidth bottlenecks in multi-agent concurrent scenarios.Experimental results demonstrate that under 20 concurrent request testing conditions,the cloud-side brain achieves video understanding inference latency of(62.4±1.2)ms and perception-decision command issuance frequency of 16 Hz;the edge-side cerebellum provides motion control inference latency of(39.2±0.8)ms and motion planning command issuance frequency of 25 Hz.Compared with traditional Ethernet architectures(average latency＞150 ms),this architecture improves latency stability by 58.4%and ensures bandwidth isolation for 20 concurrent streams through 50G-PON technology.The experimental data validate that the proposed end-edge-cloud architecture meets current concurrency requirements for intelligent agents.

Though serverless computing offers transformative benefits for deploying machine learning (ML) services, it faces challenges in meeting strict real-time service-level objectives (SLOs) of ML inference while maintaining resource efficiency. Fortunately, a new binary instruction format, WebAssembly (or Wasm), offers a promising solution for serverless ML inferences thanks to its short startup time and efficient execution compared with traditional container-based solutions. Therefore, we introduce InfSquad, a serverless ML inference framework designed to balance SLO-aware execution with resource efficiency. The key design of InfSquad is a Wasm-assisted hybrid serverless function runtime to harness the complementary strengths of Wasm and traditional containerized function runtime. Based on the hybrid runtime, InfSquad advocates an SLO-aware runtime scheduling approach that delivers efficient serverless inference. InfSquad leverages a proactive runtime recycling mechanism to increase resource efficiency further. Experiments on real-world applications show that, compared with state-of-the-art serverless systems, InfSquad achieves 19.8%-85.1% SLO violation reduction with 32.5%-74.8% fewer resources.

Serverless edge computing is attracting increasing attention due to its management-free deployment and on-demand resource provision characteristics. However, the limited resources on edge servers make efficient task scheduling crucial. Specifically, the diverse and dynamic requests in edge scenarios and the intertwined networking and computing process in serverless edge make the existing one-for-all CPU scheduling strategy fall flat. To address the CPU scheduling problem above, we introduce EdgeSched, an adaptive user-space scheduling framework for serverless functions in edge computing. EdgeSched proposes an enclave-based hybrid scheduling technique that separates network I/O tasks from diverse computational tasks efficiently to different enclaves and applies the best-fit scheduling policy on each enclave. Furthermore, EdgeSched leverages deep reinforcement learning to adaptively schedule CPU resources into different enclaves and adapts scheduling different algorithms in the user space. Through experiments on real-world edge devices and workloads, we observe a more than 2x performance improvement compared to state-of-the-art methods.

To safeguard user data privacy, on-device inference has emerged as a prominent paradigm on mobile and Internet of Things (IoT) devices. This paradigm involves deploying a model provided by a third party on local devices to perform inference tasks. However, it exposes the private model to two primary security threats: model stealing (MS) and membership inference attacks (MIA). To mitigate these risks, existing wisdom deploys models within Trusted Execution Environments (TEEs), which is a secure isolated execution space. Nonetheless, the constrained secure memory capacity in TEEs makes it challenging to achieve full model security with low inference latency. This paper fills the gap with TensorShield, the first efficient on-device inference work that shields partial tensors of the model while still fully defending against MS and MIA. The key enabling techniques in TensorShield include: (i) a novel eXplainable AI (XAI) technique exploits the model's attention transition to assess critical tensors and shields them in TEE to achieve secure inference, and (ii) two meticulous designs with critical feature identification and latency-aware placement to accelerate inference while maintaining security. Extensive evaluations show that TensorShield delivers almost the same security protection as shielding the entire model inside TEE, while being up to 25.35x (avg. 5.85x) faster than the state-of-the-art work, without accuracy loss.

On-device continual learning (CL) enables lifelong and privacy-preserving learning for various edge intelligent applications. Increasing the number of model parameters as new learning tasks emerge is effective in ensuring learning quality but inefficient in memory cost, especially for resource-constrained devices. In this paper, we introduce InfiniCL, the first ondevice CL system that dynamically balances memory cost and learning quality. A key idea behind InfiniCL is elastic continual learning: selectively freezing layers in the expanding model and periodically distilling the model, preventing unbounded memory growth while preserving learning quality for new tasks. This novel CL paradigm opens a new challenging problem: how to decide the memory allocation of the model and data to achieve better learning Quality of Service (QoS) under the limited memory budget? To alleviate this challenge, we further propose a Bayesian Optimization-driven algorithm to jointly optimize layer freezing selection and data-model memory allocation. Evaluations show that InfiniCL outperforms state-of-the-art methods on diverse memory constraints, achieving 5.34-7.15% and 2.72-9.36% higher accuracy on CIFAR-100 and ImageNet-100, respectively.

The growing popularity of encrypted container images in registries poses unique challenges for storage management due to the necessity for deduplication amidst rising image volumes. Traditional deduplication struggles with encrypted content, which inherently disguises duplicate data as distinct due to its randomized nature. Current advanced methods tackle this issue by decompressing images and applying message-locked encryption (MLE). However, these techniques face considerable challenges. Minor content changes can impair deduplication effectiveness, and decompressing layers increases storage requirements. Furthermore, this process negatively impacts both the speed at which users access the images and the overall system throughput. We propose SimEnc, a high-performance and secure deduplication system for encrypted container images by exploiting multiple similarity spaces. SimEnc pioneers the integration of semantic hashing with MLE to effectively parse semantic relationships across layers, thereby increasing deduplication efficacy. This system incorporates a rapid selection mechanism for similarity spaces, offering enhanced flexibility over previous models that relied on full decompression. By adopting Huffman decoding to navigate new similarity spaces, SimEnc not only improves deduplication ratios but also enhances overall performance. Our experimental results demonstrate that SimEnc substantially reduces storage needs by up to 261.7% compared to encrypted serverless platforms and by 54.2% against plaintext registries, while also delivering superior pull latency metrics.

The Internet of Things (IoT) has become an integral part of daily life, enabling seamless interaction between humans and the physical world. However, prototyping IoT applications remains an arduous task, requiring expertise in both hardware and software development. Current low-code and zero-code development approaches fail to address the tight coupling between hardware and software as well as the performance of the generated application, limiting their applicability. We introduce Jarvis, a zero-code prototyping framework for IoT applications that leverages composable hardware-software abstractions. By abstracting physical constraints and introducing a parameter-free self-reflection generation mechanism, Jarvis enables large language models to understand and address the coupling constraints in IoT development with minimized token cost and hallucination. Jarvis also considers the cost and energy efficiency of the resulting IoT prototype, optimizing the performance of the generated solutions. Evaluations demonstrate that Jarvis outperforms state-of-the-art methods, reducing hardware costs by 23.1%-75.5%, saving power consumption by 18.7%-95.2%, and lowering token usage for prototyping by 32.1%-86.2%.

Deploying large language models (LLMs) with low-rank adaptation (LoRA) on mobile devices is promising due to their capability to complete diverse domain-specific tasks while ensuring privacy and accessibility. In this paper, we introduce MobiLoRA to accelerate LoRA-based LLM inference on mobile devices. MobiLoRA focuses on optimizing the key-value (KV) caches due to the limited computing and memory resources of mobile devices. The key insight of MobiLoRA lies in the utilization of two contexts for on-device LoRA serving: semantic-level contexts, such as prompts with shared prefixes, and system-level contexts, such as the application status (e.g., foreground or killed) of LLM requests. Specifically, for semantic-level contexts, MobiLoRA proposes similarity-aware delta encoding, which leverages token-wise similarity in KV caches across LoRA adapters for efficient storage and reuse. Furthermore, MobiLoRA advocates context-aware KV cache management to optimize cache retention and eviction considering the system-level contexts. We fully implement MobiLoRA and compare it with state-of-the-art LLM serving frameworks using real-world mobile device traces. Results show that MobiLoRA accelerates LoRA-based LLM inference by 57.6% on mobile devices.

Nowadays, there is a growing trend to deploy machine learning (ML) models on edge devices. To cope with the increasing resource requirements of current ML models, multi-accelerator edge devices that integrate CPU, GPU, NPU, or TPU in a single SoC gain popularity. However, we observe that existing ML inference serving frameworks are poor in utilizing the unique hardware architecture of these edge devices. In this paper, we present INFSCALER, an efficient ML inference serving framework tailored for multi-accelerator edge devices. INFSCALER discovers the architectural bottleneck of ML models and designs a bottleneck-aware asymmetric auto-scaling technique to facilitate efficient resource allocation for ML models on the edge. Furthermore, INFSCALER capitalizes on the hardware's unified memory feature inherent to edge devices, ensuring efficient data sharing between the asymmetrically scaled model partitions. Our experimental results show that INFSCALER achieves up to 126.59% throughput improvement and 27.32% resource reduction while satisfying the latency requirements compared with the state-of-the-art inference serving approaches.

In-browser neural network inference offers the promise of cross-platform AI applications, but faces severe latency and energy challenges on resource-constrained devices. In this paper, we present nnWeb, a WebGPU-based in-browser neural network inference framework with optimized latency and energy efficiency. nnWeb dynamically partitions neural network and facilitates the collaborative offloading between client browser and server. nnWeb operates in two phases: (1) layer-wise isolation-based profiling, which is used to predict per-layer execution latency and energy on heterogeneous hardware; and (2) asynchronous execution-based DNN partitioning, which continuously monitors network bandwidth and device load to select the optimal partition point using WebGPU’s native pipeline parallelism, minimizing total latency or energy consumption by solving a closed-form optimization at runtime. Extensive evaluation on various in-browser AI models and networking conditions shows that nnWeb achieves an average reduction of 30% to 52% in total inference latency compared with static partitioning. Moreover, nnWeb realizes energy savings ranging from 11.3% to 44.0% in contrast to standalone browser inference.

With the development of sensing, wireless communication, and real-time computing technologies, vehicles are gradually becoming more and more intelligent. To provide safe autonomous mobility services, connected autonomous vehicles (CAVs) need to obtain complete information about their environment and process it in real-time to make driving decisions. However, the rapid increase in data volume puts pressure on CAVs to process tasks in real time. This survey analyzes CAVs delay optimization from the perception layer, communication layer, computation layer, and cross-layer. According to different coordination modes, each layer of CAVs is divided, and the problem of delay optimization is classified in fine granularity. This survey will help researchers gain insight into the mechanism of delay optimization on CAVs and highlight the key role of optimized delay in autonomous driving.

Web of Things (WoT) is an emerging concept to connect IoT devices to the web using standard interfaces. This provides interoperability between different IoT platforms and enables seamless integration with web and cloud services. However, running sophisticated web services directly on resource-constrained IoT devices is challenging due to limitations in memory, computation, and energy. This paper proposes WaWoT, a Wasm-based framework for flexible and efficient Web of Things services. WaWoT allows flexible WoT service development using annotations and automatic partitioning. It also enables dynamic service migration using WebAssembly modules to adapt placement between IoT devices and web clients. We also introduce an ahead-of-time compiler optimized for low memory usage through techniques like streamed compilation and trimming. For energy efficiency, we use optimizations like bulk instruction writing and direct I/O accessing. Safety is ensured through compile-time and run-time analyses to guarantee sandboxed execution. Evaluations demonstrate WaWoT exhibits better flexibility than existing WoT development approaches. Furthermore, WaWoT can also reduce RAM usage by 84.9x and energy consumption by 1.9x-4.9x over existing WebAssembly runtimes. Overall, it enables efficient, safe, and flexible WoT services on constrained IoT devices.