On-device LLM inference is increasingly attractive for privacy-preserving, reliable, and cost-effective deployment, yet its energy and thermal costs remain a critical bottleneck. Existing systems primarily optimize for decoding speed, implicitly assuming that faster execution is always preferable. We show instead that on-device LLM inference often has exploitable configuration slack: modestly lowering NPU and memory frequencies preserves quality of experience (QoE) while substantially improving energy efficiency and reducing heat. Realizing this opportunity in production is challenging. The most energy-efficient NPU/DDR setting varies with the model, inference engine, platform, and runtime conditions, with no stable ranking across configurations. Commercial devices further lack component-level power sensing, and shell temperature evolves with request arrivals, response lengths, and thermal history. To address these challenges, we propose EnerInfer, the first on-device LLM inference framework that jointly manages energy efficiency, throughput, and thermal comfort for LLM workloads. EnerInfer replaces per-model profiling and sensor-heavy control with disaggregated, model-structure-aware prediction and ranking-driven online feedback. It predicts throughput and power for unseen LLMs across NPU/DDR frequency settings, selects QoE-satisfying efficient configurations under runtime interference, and uses lightweight limited-horizon thermal prediction to dynamically switch between energy-optimized and thermally constrained inference. Evaluations on real-world LLMs show that EnerInfer improves energy efficiency by up to 65

As large language models (LLMs) move from research to production, understanding how inference engines behave in real time has become both essential and elusive. Unlike general-purpose engines such as ONNX Runtime, today's LLM inference systems offer little operator-level visibility, leaving developers blind to where time and resources go. Even basic questions – is this workload memory-bound or compute-bound? – often remain unanswered. To close this gap, we develop a fine-grained, non-intrusive profiling framework for modern LLM inference engines, exemplified by llama.cpp but applicable to similar runtime architectures. Built on extended Berkeley Packet Filter (eBPF) technology, our system dynamically attaches probes to runtime functions across multiple layers – without modifying or recompiling the source. It transforms collected traces into rich visualizations of operators, graphs, timelines, and hardware counter trends, exposing how dense inference, Mixture-of-Experts routing, and operator offloading behave in practice. With less than 4

Memory bandwidth regulation and cache partitioning are widely used techniques for achieving predictable timing in real-time computing systems. Combined with partitioned scheduling, these methods require careful co-allocation of tasks and resources to cores, as task execution times strongly depend on available allocated resources. To address this challenge, this paper presents a 0-1 linear program for task-resource co-allocation, along with a multi-objective heuristic designed to minimize resource usage while guaranteeing schedulability under a preemptive EDF scheduling policy. Our heuristic employs a multi-layer framework, where an outer layer explores resource allocations using Pareto-pruned search, and an inner layer optimizes task allocation by solving a knapsack problem using dynamic programming. To evaluate the performance of the proposed optimization algorithm, we profile real-world benchmarks on an embedded AMD UltraScale+ ZCU102 platform, with fine-grained resource partitioning enabled by the Jailhouse hypervisor, leveraging cache set partitioning and MemGuard for memory bandwidth regulation. Experiments based on the benchmarking results show that the proposed 0-1 linear program outperforms existing mixed-integer programs by finding more optimal solutions within the same time limit. Moreover, the proposed multi-objective multi-layer heuristic performs consistently better than the state-of-the-art multi-resource-task co-allocation algorithm in terms of schedulability, resource usage, number of non-dominated solutions, and computational efficiency.

Directed acyclic graph (DAG) tasks are currently adopted in the real-time domain to model complex applications from the automotive, avionics, and industrial domains that implement their functionalities through chains of intercommunicating tasks. This paper studies the problem of scheduling real-time DAG tasks by presenting a novel schedulability test based on the concept of trivial schedulability . Using this schedulability test, we propose a new DAG scheduling framework ( edge generation scheduling—EGS ) that attempts to minimize the DAG width by iteratively generating edges while guaranteeing the deadline constraint. We study how to efficiently solve the problem of generating edges by developing a deep reinforcement learning algorithm combined with a graph representation neural network to learn an efficient edge generation policy for EGS. We evaluate the effectiveness of the proposed algorithm by comparing it with state-of-the-art DAG scheduling heuristics and an optimal mixed-integer linear programming baseline. Experimental results show that the proposed algorithm outperforms the state-of-the-art by requiring fewer processors to schedule the same DAG tasks. https://github.com/binqi-sun/egs

Typically, in a cyber-physical system (CPS), timing jitters between sensing and actuation adversely affect its physical behavior. Logical execution time (LET) paradigm has gained industry attention because it offers zero jitters in tasks’ response time. In autonomous CPS such as self-driving cars, Robot Operating System 2 (ROS2) is becoming a popular software platform to implement computation tasks. Towards a practical LET implementation of ROS2 computation chains, we propose to use at least two table-driven reservation servers: (i) one exclusively runs the processor thread that publishes the chain’s output while (ii) others isolate the chain’s main computations from best-effort workloads on the same processing unit. We show how to architecturally adapt the chain non-intrusively as well as dimension the servers and allocate appropriate threads to them so that we obtain negligible jitters while keeping the observed maximum end-to-end latency comparable to a high-priority execution of the chain. Also, our approach is versatile and can produce latency bands, thereby offering an opportunity to co-optimize jitters and average end-to-end latency. This further supports multi-mode application design, which is especially important for a high-performance operation in different environments, e.g., city and highway driving. Our idea does not involve modification or recompilation of the application code or ROS2 libraries, which is crucial in industry settings. We have also developed tools to automate our approach. By applying our proposed mechanism on a real-world benchmark implementing Lidar-based localization, we maintain a constant end-to-end latency that is even 13% shorter than an improved LET implementation.

Autonomous applications are typically developed over Robot Operating System 2.0 (ROS2) even in time-critical systems like automotive. Recent years have seen increased interest in developing model-based timing analysis and schedule optimization approaches for ROS2-based applications. To complement these approaches, we propose a tracing and measurement framework to obtain timing models of ROS2-based applications. It offers a tracer based on extended Berkeley Packet Filter that probes different functions in ROS2 middleware and reads their arguments or return values to reason about the data flow in applications. It combines event traces from ROS2 and the operating system to generate a directed acyclic graph showing ROS2 callbacks, precedence relations between them, and their timing attributes. While being compatible with existing analyses, we also show how to model (i) message synchronization, e.g., in sensor fusion, and (ii) service requests from multiple clients, e.g., in motion planning. Considering that, in real-world scenarios, the application code might be confidential and formal models are unavailable, our framework still enables the application of existing analysis and optimization techniques. We demonstrate our framework's capabilities by synthesizing the timing model of a real-world benchmark implementing LIDAR-based localization in Autoware's Autonomous Valet Parking.

Autonomous driving is a big challenge for next-generation vehicles and requires multiple computationally-intensive deep neural networks (DNNs) to be implemented on distributed automotive platforms. Distributed software-enabling autonomous functionalities-has strict timing requirements, e.g., low and deterministic end-to-end latency. Such timings rely on the communication technologies used in the automotive platform, as much on the computation performance of CPUs, GPUs, TPUs, and FPGAs. Hence, we advocate the use of Remote Direct Memory Access (RDMA) technology-typically used in data centers-in automotive platforms. As shown by our experiments with real hardware, Soft-RoCE (software implementation of RDMA) offers low latency communication because of minimal CPU involvement and reduced memory copies. Simultaneously, we show that the native implementation of RDMA does not support determinism, i.e., there is a high variation in communication delays in the presence of interfering data packets. To mitigate this issue, we propose a multi-layer communication stack comprising a deterministic scheduler on top of the Soft-RoCE layer. Further, we have developed a C++ library that offers easy-to-use communication interfaces for distributed applications while implementing the proposed architecture. Experiments show that our library (i) reduces the end-to-end latency of distributed object detection by nearly 9% while having an implementation overhead of less than 1.5% and (ii) minimizes the effects of other data traffic on the delay in high-priority communication.

Cache partitioning techniques have been successfully adopted to mitigate interference among concurrently executing real-time tasks on multi-core processors. Considering that the execution time of a cache-sensitive task strongly depends on the cache available for it to use, co-optimizing cache partitioning and task allocation improves the system's schedulability. In this paper, we propose a hybrid multi-layer design space exploration technique to solve this multi-resource management problem. We explore the interplay between cache partitioning and schedulability by systematically interleaving three optimization layers, viz., (i) in the outer layer, we perform a breadth-first search combined with proactive pruning for cache partitioning; (ii) in the middle layer, we exploit a first-fit heuristic for allocating tasks to cores; and (iii) in the inner layer, we use the well-known recurrence relation for the schedulability analysis of non-preemptive fixed-priority (NP-FP) tasks in a uniprocessor setting. Although our focus is on NP-FP scheduling, we evaluate the flexibility of our framework in supporting different scheduling policies (NP-EDF, P-EDF) by plugging in appropriate analysis methods in the inner layer. Experiments show that, compared to the state-of-the-art techniques, the proposed framework can improve the real-time schedulability of NP-FP task sets by an average of 15.2% with a maximum improvement of 233.6% (when tasks are highly cache-sensitive) and a minimum of 1.6% (when cache sensitivity is low). For such task sets, we found that clustering similar-period (or mutually compatible) tasks often leads to higher schedulability (on average 7.6%) than clustering by cache sensitivity. In our evaluation, the framework also achieves good results for preemptive and dynamic-priority scheduling policies.

The core idea of flexible manufacturing is adapting to changes. In this domain, the machine is not confined to a single fixed type of process but can perform different jobs (e.g., cutting, drilling) in different ways (e.g., varying speed, tool, power consumption). This adaptability should be enabled by a detailed view of how the machines work. The idea is to perform machine scheduling by exploiting the dynamical models-expressed as differential equations-of manufacturing processes, i.e., both machines and production items. The main innovation in this paper is the ability to compute a machine's schedule where the state of the product does not linearly evolve in time but is determined by the set of differential equations instead. Finding the schedule is defined as a multi-objective optimization problem-manufacturers may seek a trade-off between processing time, energy consumption, and other cost functions. The proposed optimization is evaluated using accurate process models, exemplifying how it works and harnesses the expressiveness of differential equations.

A network switch implements multiple services and each service is formed by a number of match-action operations through several pipeline stages. These services running in the switch equipment are to process various packets based on standard internet protocols to decide the route of each packet. Data packets come in serial to a port, where each packet is processed by a service according to the contents of the packet headers and then send out via another port. Design of the switch, i.e., mapping services to physical resources in the pipeline stages, aims to achieve low switching latency with small chip area while respecting data-flow dependencies and hardware constraints. The current practice relies on expertise of engineers empirically, which is laborious and generates mediocre results. In this paper, we propose a switching pipeline design optimizatton technique, called SPOT. Our main contributions are as follows: (i) We first formulate the bi-objective (latency and chip area) constrained design optimization problem; (ii) SPOT quickly spots a feasible solution from a largely unfeasible design space using a dependency-aware greedy algorithm; (iii) Based on the above feasible seed, SPOT explores the design space with hundreds of decision dimensions towards Pareto optimal solutions using non-dominated sorting genetic algorithm II (NSGA-II) and multi-objective tabu search (MOTS), both adapted to be deployed in this problem setting. We apply SPOT on three sets of real-world network services. In comparison to the design sheets prepared by expert engineers, experiments show that SPOT offers 20.63% shorter service latency and 4.55% smaller chip area on average. As a by-product, the power consumption is lowered by 23.72% on average, which is correlated to the chip area. For hard real-time scenarios, the longest service latency a data packet may experience is the major concern. SPOT reduces the worst-case service latency by 12.65% on average. SPOT is the first automated optimization solution for switching pipeline design in network chips, being utilized in millions of network products of various kinds and saving manual efforts from days to minutes.

This paper studies the following basic flexible manufacturing problem: Given N machines that can perform the same job on a production item (e.g., drilling or tapping) but with different capabilities (e.g., energy requirements and speeds), what is an optimal schedule for the job on these machines? While this is a well-studied problem, the main innovation this paper introduces is the explicit modeling of the underlying process dynamics—i.e., the physical interaction of the item and the machine—using differential equations. The resulting scheduling problem is in a hybrid systems setting that involves determining the transition times between states, where the system evolution in each state is defined by differential equations. To the best of our knowledge, such a cyber-physical systems (CPS) oriented approach to machine scheduling has not been studied before, although it lies at the core of flexible manufacturing in Industry 4.0. We believe that this new formulation might lead to a renewed interest in machine scheduling problems, but now in a hybrid/CPS-oriented setting.

This paper is concerned with the following question: Given a set of control tasks that are not schedulable, i.e., their required timing properties cannot be satisfied, what should be changed? While the real-time systems literature proposes many different schedulability analysis techniques, it surprisingly provides almost no guidelines on what should be changed to make a task set schedulable, when it is not. We show that when the tasks in question are control tasks, this timing debugging question in the context of cyber-physical systems (CPS) may be answered by exploiting the dynamics of the physical systems that these control tasks are expected to influence. Towards this, we study a very simple setup, viz., when a set of periodic tasks with implicit deadlines is not schedulable, by how much should the periods be changed in order to make the task set schedulable? Among the many ways in which the periods can be modified, our proposed strategy is to change the periods in a manner such that while the task set becomes schedulable, the poles of the closed-loop system experience the minimal shift. Since the poles influence the closed loop dynamics of the system, we thereby ensure that we obtain a system with the desired timing properties whose dynamics is very similar to the dynamics of the original (non-schedulable) system. We formulate this CPS timing debugging strategy as an optimization problem and illustrate it with a concrete example.

Controller design and their software implementations are usually done in isolated design spaces using respective COTS design tools. However, this separation of concerns can lead to long debugging and integration phases. This is because assumptions made about the implementation platform during the design phase—e.g., related to timing—might not hold in practice, thereby leading to unacceptable control performance. In order to address this, several control/architecture co-design techniques have been proposed in the literature. However, their adoption in practice has been hampered by the lack of design flows using commercial tools. To the best of our knowledge, this is the first article that implements such a co-design method using commercially available design tools in an automotive setting, with the aim of minimally disrupting existing design flows practiced in the industry. The goal of such co-design is to jointly determine controller and platform parameters in order to avoid any design-implementation gap , thereby minimizing implementation time testing and debugging. Our setting involves distributed implementations of control algorithms on automotive electronic control units ( ECUs ) communicating via a FlexRay bus. The co-design and the associated toolchain Co-Flex jointly determines controller and FlexRay parameters (that impact signal delays) in order to optimize specified design metrics. Co-Flex seamlessly integrates the modeling and analysis of control systems in MATLAB/Simulink with platform modeling and configuration in SIMTOOLS/SIMTARGET that is used for configuring FlexRay bus parameters. It automates the generation of multiple Pareto-optimal design options with respect to the quality of control and the resource usage, that an engineer can choose from. In this article, we outline a step-by-step software development process based on Co-Flex tools for distributed control applications. While our exposition is automotive specific, this design flow can easily be extended to other domains.

Feedback control loops are ubiquitous in any autonomous system. The design flow for any controller starts by determining a control strategy, while abstracting away all implementation details. However, when designing controllers for autonomous systems, there is significant computation associated with the perception modules. For example, this involves vision processing using deep neural networks on multicore CPU+accelerator platforms. Such computation can be organized in many different ways, with each choice resulting in very different sensor-to-actuator delays and tradeoffs between cost, delay, and accuracy. Further, each of these choices requires the control strategy to be designed accordingly. It is not possible for a control designer to enumerate and account for all of these choices manually, or abstract them away as "implementation details" as done in traditional controller design. In this paper we outline this problem and discuss how automated controller-synthesis techniques could help in addressing it.

CAN-FD (CAN with flexible data rate) has been developed to support automated driving as a high-bandwidth version of the conventional CAN (controller area network) bus protocol. Due to the complexity of the emerging automotive functionalities, there exist dependencies between the tasks and thus also between the CAN-FD messages. The current industrial practice is that the same application has exactly the same message transmission flow (i.e., the same ordered sequence of messages to be transmitted) across all vehicles. This renders large-scale attacks possible and potentially leads to millions of vehicles to be recalled, as one vehicle being compromised exposes all the others. To address this issue, an application could have different (obfuscated) message flows on individual vehicles. The challenge is to find a large number of available flows (i.e., flows that respect dependencies and meet application deadlines) within short time. For this purpose, we propose a novel priority assignment approach, which assigns the ordered positions in a flow (named priorities) to the messages. It dynamically generates new valid flows (i.e., flows with only dependencies respected and deadlines not considered) by message swapping, instead of exploring all valid flows as in the existing approaches. We apply pruning through affix-matching to further enhance the efficiency. That is, the prefix, infix, and suffix are all matched when determining whether a certain flow should be discarded without evaluating its availability, aiming for lower false positive rate (FSR) and false negative rate (FNR) than adfix-matching (only prefix and suffix are matched) in the state-of-the-art approach. Experimental results show that the proposed approach dominates the state-of-the-art approach, in the number of available flows found (up to 79x) and time consumption (up to 200x), most notably when the proportion of available flows is small. This work is an important step for obfuscated priority assignment to be deployed on practical CAN-FD messages.

While wired networks provide a reliable platform for networked cyber-physical systems (CPS), there is an increasing demand for CPS built upon wireless networks. However, wireless connectivity also implies varying and unpredictable end-to-end delays due to packet loss, interference by concurrently transmitting nodes or the necessity to forward packets via one or many intermediate nodes. This is typically accounted for by designing controllers for the worst-case end-to-end delay. This guarantees stability also when the largest possible delay occurs. However, the delays observed during normal operation are significantly below the worst-case. As a result of the overly pessimistic controller design, the control performance becomes unnecessarily low. In this work, for the first time, we present a generic technique to handle varying end-to-end delays in wireless CPS. While maintaining a stable operation, our technique preserves a high control performance. In essence, we propose a proactive feedback strategy that computes future control inputs for different possible delays a priori and sends them to the actuator in a single packet. When new control inputs are delayed, pre-computed ones accounting for higher delays are applied at appropriate actuation instants. In this way, a controller responds fast when control input arrives with low latencies, while adaptively acting more conservatively when packets are delayed. Our proposed strategy is independent of the controller design technique and the communication protocol used. We also present a real-world implementation of our proposed technique on a physical testbed. Experiments suggest that the proposed strategy improves the control performance of the system by up to 63 % compared to existing control schemes.

In practice, safety-critical cyber-physical systems (CPS) are often implemented using high quality-of-service (QoS) resources to provide maximum performance in all scenarios. Such implementations are oblivious to the changing criticality levels of CPS based on their physical dynamics (e.g., steady or transient state). Considering that high-QoS resources are constrained for cost-sensitive CPS, such criticality-oblivious implementations are highly inefficient. Towards a tighter dimensioning of these resources, state-of-the-art approaches have considered multi-QoS resources and studied criticality-aware dynamic resource allocation along the lines of mixed-criticality systems. However, these approaches have high implementation overheads. Moreover, in safety-critical domains like automotive and avionics, certification of such dynamic policies is challenging and the implementation platforms typically do not support dynamic reconfiguration. To address these challenges, we present GoodSpread that uses a static scheduling strategy and offers the same performance guarantees while saving resources (more than 50 % in certain cases) compared to the existing dynamic schemes. The main idea here is to spread the high-QoS resources as uniformly as possible over time in order to accommodate the uncertainty of when the criticality level might change. Our proposed strategy studies the physical dynamics to determine the spread factor, i.e., how often the high-QoS resources need to be provisioned. We further propose an extensibility-driven optimization approach to obtain a static schedule that will accommodate future workloads on the remaining resources with maximum flexibility.

Automotive software implements different functionalities as multiple control applications sharing common platform resources. Although such applications are often developed independently, the control performance of the resulting system depends on how these applications are integrated. A key integration challenge is to efficiently schedule these applications on shared resources with minimal control performance degradation. We formulate this problem as that of scheduling multiple distributed periodic control tasks that communicate via messages with non-zero jitter. The optimization criterion used is a piecewise linear representation of the control performance degradation as a function of the end-to-end latency of the application. The three main contributions of this article are: 1) a constraint programming (CP) formulation to solve this integration problem optimally on time-triggered architectures; 2) an efficient heuristic called Flexi; and 3) an experimental evaluation of the scalability and efficiency of the proposed approaches. In contrast to the CP formulation, which for many real-life problems might have unacceptably long running times, Flexi returns nearly optimal results (0.5 percent loss in control performance compared to optimal) for most problems with more acceptable running times.

Modeling and design of automotive systems from a cyber-physical system (CPS) perspective have lately attracted extensive attention. As the trend towards automated driving and connectivity accelerates, strong interactions between vehicles and the infrastructure are expected. This requires modeling and control of the traffic network in a similarly formal manner. Modeling of such networks involves a tradeoff between expressivity of the appropriate features and tractability of the control problem. Back-pressure control of traffic signals is gaining ground due to its decentralized implementation, low computational complexity, and no requirements on prior traffic information. It guarantees maximum stability under idealistic assumptions. However, when deployed in real traffic intersections, the existing back-pressure control algorithms may result in poor junction utilization due to (i) fixed-length control phases; (ii) stability as the only objective; and (iii) obliviousness to finite road capacities and empty roads. In this paper, we propose a CPS-oriented model of traffic intersections and control of traffic signals, aiming to address the utilization issue of the back-pressure algorithms. We consider a more realistic model with transition phases and dedicated turning lanes, the latter influencing computation of the pressure and subsequently the utilization. The main technical contribution is an adaptive controller that enables varying-length control phases and considers both stability and utilization, while taking both cases of full roads and empty roads into account. We implement a mechanism to prevent frequent changes of control phases and thus limit the number of transition phases, which have negative impact on the junction utilization. Microscopic simulation results with SUMO on a 3 × 3 traffic network under various traffic patterns show that the proposed algorithm is at least about 13% better in performance than the existing fixed-length backpressure control algorithms reported in previous works. This is a significant improvement in the context of traffic signal control.

Active cell balancing is performed to minimize the variation in the charge levels of the individual cells in a high-power battery pack, to improve its usable capacity. The process of charge equalization is carried out by scheduling pairs of cells to transfer charge over a hardware circuit. Improving the time for charge equalization has been studied in the power electronics and the electronic design automation domains. However, these approaches have focused on the electronics issues and used heuristics to determine the charge transfer schedule. Hence, no optimality results on charge equalization times are known. We, for the first time, take a real-time systems approach and propose an optimal scheduling framework for active cell balancing. The proposed framework employs a hybrid optimization technique consisting of two sequential stages. In the first stage, we solve a mixed-integer linear programming problem to identify the time-optimal set of charge transfers required to achieve charge equalization. In the second stage, we construct a conflict graph based on the obtained charge transfers, to which we apply the minimum vertex coloring algorithm to synthesize the minimum length schedule. Results show that our proposed framework can reduce the charge equalization time by more than 50% (e.g., from 11 h to 5h). Hence, this has real benefits, e.g., in the context of charging electric vehicles. While task and message scheduling problems have been extensively studied in the real-time systems literature, the scheduling problem we study here, has not been addressed before.