Large language models (LLMs) have recently achieved remarkable performance in text generation, capturing the attention of a broad audience. This success, driven by the rapid growth in model parameters, comes at the expense of significantly higher operational costs and decreased processing speed. These costs, combined with privacy concerns around cloud-based deployments, have motivated research into running LLMs on commodity hardware. For example, researchers have used the memory hierarchy to boost throughput by increasing the number of batches. These studies, however, tend to overlook or inefficiently utilize the additional computational resources provided by the CPU. In this work, we present a dynamic workload allocation technique that efficiently distributes computation across all available hardware resources. The proposed method targets decoder-based models on standard general-purpose hardware, effectively minimizing idle periods for both the CPU and the GPU. Experiments show that our approach achieves up to 30% higher throughput compared to the state of the art, regardless of model architecture, LLM optimizations, and input batch sizes.

Data centers employ live virtual machine (VM) migration to optimize resource usage while ensuring continuous execution of guest operating systems. Given the current resource utilization, sophisticated algorithms determine when and where to migrate which VMs. Surprisingly little attention, however, is given to selecting the appropriate migration technique based on specific host and guest workload characteristics. This work first shows that relying on a single live migration algorithm leads to significantly more Service-Level Objective (SLO) violations and higher resource usage than adaptively selecting the most suitable migration algorithm. Building on this observation, we then present an intelligent live migration framework that selects the most appropriate live migration algorithm based on SLOs and operational cost factors, using a multi-objective optimization approach. Through a comprehensive evaluation across diverse hotspot and consolidation scenarios, we demonstrate that the presented framework is able to substantially reduce SLO violation while optimizing key operational metrics. The framework reduces the total migration time by a factor of 1.5 and decreases SLO violations by nearly an order of magnitude compared to the predominantly used pre-copy method. Moreover, it achieves near-optimal VM migration technique selection compared to an Oracle under varying workload conditions. The results indicate that intelligent selection of live migration algorithms can significantly enhance both application performance and resource efficiency in virtualized environments.

This study presents a mathematical model to analyze the competition between smart and traditional home appliance manufacturers in Iran and South Korea using a game theory approach. The smart manufacturer utilizes IoT-enabled products and a Sensing-as-a-Service (SaaS) ecosystem to collect and analyze real-time operational data, enabling predictive maintenance, energy optimization, and intelligent product design. This data-driven approach extends the product life cycle, reduces operating costs, and reduces material waste and carbon emissions. In contrast, the traditional manufacturer relies on conventional R&D and after-sales feedback, which limits responsiveness and efficiency. The proposed model is the first to highlight the strategic value of data analytics in the SaaS ecosystem in shaping competitive advantage, sustainability, and market performance by considering customer behavior and government incentives. The results show that providing targeted incentives for data sharing significantly increases demand, profitability, and environmental benefits, especially in mature digital markets. Furthermore, the study shows that supporting context-specific policies enhances the effectiveness of smart manufacturing strategies. Overall, this research provides actionable insights for policymakers, manufacturers, and IoT stakeholders seeking to foster sustainable, competitive, and digital industrial systems in emerging and advanced economies.

The proliferation of Artificial Intelligence (AI) applications has driven a substantial demand for deploying Neural Networks (NNs) across a wide range of device platforms. Modern networks have become so computationally intensive that even relatively simple Convolution Neural Networks (CNNs) often require costly specialized hardware for efficient execution. To mitigate this challenge, techniques that distribute inference workloads across interconnected, resource-constrained devices have become increasingly important. While existing approaches typically rely on empirical models or support only limited partitioning dimensions, we introduce ADaPS-FC, a novel framework for optimally distributing CNN inference workloads across heterogeneous embedded devices. Our analytical model partitions the height, width, and channel dimensions of 4D CNN tensors while also exploring partition configurations that enable layer fusion. In addition, it introduces weight-division strategies for fully connected layers and small output operators to improve both memory usage and latency, fully accounting for each device’s computational capabilities and inter-device communication overhead. This work extends the previously proposed ADaPS method by addressing the limitations encountered in the later, non-divisible layers of CNN models. To efficiently navigate the large search space of possible partitionings, ADaPS-FC employs a hybrid optimization algorithm that combines Alpha–Beta pruning with dynamic programming for the earlier layers, where height or width division is feasible. For the later layers, where the prior method was unable to apply spatial partitioning, the framework introducess weight (channel) division to enable further distribution opportunities. We evaluate ADaPS-FC across multiple CNN architectures deployed on interconnected heterogeneous hardware, using configurations ranging from two to four devices. Experimental results show that ADaPS-FC improves inference time by 1.3 × on average over previous state-of-the-art.

Parametrizable neural network accelerators enable the deployment of targeted hardware for specialized environments. Finding the best architecture configuration for a given specification, however, is challenging. A large number of hardware configurations have to be considered, and for each hardware instance, an efficient software execution plan needs to be found, leading to a vast search space. Prior work has tackled this problem by dividing the search into subproblems for individual layers of a network. There is no guarantee, however, that the overall best hardware configuration that delivers the desired end-to-end performance across the entire network is among the best individual layer configurations. This work presents SENNA, a unified hardware/software space exploration framework for parametrizable neural network accelerators. To guide the exploration toward the overall best configuration, SENNA employs a multi-objective genetic algorithm with a novel design space representation that encodes the configuration of hardware and software parameters in a single chromosome. Using the Parallel Island Model (PIM), each layer is represented by one or more individual islands each containing a separate population to simultaneously search for the best configuration across the entire network. A tailored gene migration technique enables the exchange of genes between the populations of different islands. SENNA is evaluated with three parametrizable architectures and four neural networks. The evaluation result demonstrates that SENNA achieves upto 1.92x EDP improvement compared to the State-of-the-Art. With equivalent evaluation budgets, SENNA shows 2.5x-9.3x speedup compared to an Oracle scheme and the State-of-the-Art.

The growing adoption of AI applications has led to an increased demand for deploying neural networks on diverse device platforms. However, even modest networks now require specialized hardware for efficient execution due to their rising computational cost. To address this, distributed execution across connected, resource-constrained devices is gaining importance. While prior work relies on empirical models or supports limited partitioning, we present ADaPS, a novel framework for distributing Convolutional Neural Networks (CNNs) inference workloads across heterogeneous embedded devices. Our analytical model partitions the height and width dimensions of 4D tensors and explores layer fusion opportunities, accounting for compute, memory, and communication constraints. ADaPS efficiently explores the vast partitioning space using a tree-based hybrid optimization algorithm combining Alpha-Beta pruning and dynamic programming. Evaluations on multiple CNNs and device configurations show that ADaPS is able to improve inference latency by up to 1.2x on average while significantly reducing data transfers compared to state-of-the-art methods.

Large language models (LLMs) have recently captured the attention of a broad audience. To a large part, their exceptional performance in text generation was made possible by an exponential growth of the model parameters. This growth, however, comes at the expense of significantly higher operational costs and a decreased processing speed. Recent research has focused on running LLMs on commodity hardware, for example, by employing the memory hierarchy to augment throughput by increasing the number of batches. These studies, however, tend to overlook or inefficiently utilize the additional computational resources provided by the CPU. In this paper, we introduce a technique capable of efficiently harnessing all available computational resources through a finely tuned and dynamic workload allocation approach. This technique applies to decoder-based models on standard general-purpose hardware, effectively minimizing idle periods for both the CPU and the GPU. We conducted experiments involving various large language models, each representing distinct decoder-based architectures. Compared to the state-of-the-art, the results demonstrate a potential for an increase of up to 105% in throughput with the OPT-30B model.

Smart vehicles have a rising demand for computation resources, and recently vehicular edge computing has been recognized as an effective solution. Edge servers deployed in roadside units are capable of accomplishing tasks beyond the capacity which is embedded inside the vehicles. However, the main challenge is to carefully select the tasks to be offloaded considering the deadlines, and in order to reduce energy consumption, while delivering a good performance. In this paper, we consider a vehicular edge computing network in which multiple cars are moving at non-constant speed and produce tasks at each time slot. Then, we propose a task offloading algorithm, aware of the vehicle’s direction, based on Rainbow, a deep Q-learning algorithm combining several independent improvements to the deep Q-network algorithm. This is to overcome the conventional limits and to reach an optimal offloading policy, by effectively incorporating the computation resources of edge servers to jointly minimize average delay and energy consumption. Real-world traffic data is used to evaluate the performance of the proposed approach compared to other algorithms, in particular deep Q-network, double deep Q-network, and deep recurrent Q-network. Results of the experiments show an average reduction of 18% and 15% in energy consumption and delay, respectively, when using the proposed Rainbow deep Q-network based algorithm in comparison to the state-of-the-art. Moreover, the stability and convergence of the learning process have significantly improved by adopting the Rainbow algorithm.

Collecting and analysing large amounts of data in the Cloud-to-Edge computing continuum raises novel challenges that traditional centralised orchestration solutions cannot handle efficiently. To overcome the limitations of current centralised application management approaches, this paper presents a fully decentralised application-level orchestrator, based on the notion of self-organised interdependent Swarms. Application microservices are managed in a dynamic Orchestration Space by decentralised Orchestration Agents, governed by distributed intelligence that provides matchmaking between application requirements and resources, and supports the dynamic self-organisation of Swarms. Knowledge and trust, essential for the operation of the Orchestration Space, are managed through blockchain-based trusted solutions and the utilisation of emerging methods such as Self-Sovereign Identities (SSI) and Distributed Identifiers (DID). End-to-end security of the overall system is assured by utilising state-of-the-art cryptographic and privacy-preserving data analytics algorithms. A digital twin, that runs in parallel to the physical system, further improves its behaviour with predictive feedback. The presented concept is going to be implemented in the EU-funded Swarmchestrate project that starts in 2024.

Virtual reality has become a new option to inform the customers about product before purchasing. However, providing virtual reality may create new challenges. For instance, consumers may obtain essential information about products by using virtual reality option, but eventually they buy their products from the offline channel. This phenomenon is called webrooming. Another challenge is cyber-attacks. E-tailers and their consumers face risks from cyber-attacks. Thus, e-tailers are investing to improve cybersecurity. We consider a dual-channel supply chain consisting of an offline retailer and an e-tailer who purchase the same product from a manufacturer at the same wholesale price. The e-tailer offers a partial refund policy in order to attract the customers. Also, to reduce consumer valuation uncertainty, the e-tailer faces the decision on whether to introduce virtual reality. We analyze three scenarios. Firstly, the e-tailer does not offer virtual reality. Secondly, the e-tailer offers virtual reality, but he does not invest on cybersecurity. Thirdly, the e-tailer offers virtual reality and invests on cybersecurity.

Despite recent attempts to reduce the number of parameters of large language models (LLMs), their parameter data is still too large to fit into a single GPU. With the emergence of throughput-oriented tasks, high-throughput generative inference frameworks for LLMs on a single commodity GPU leverage GPU, DRAM, and NVMe to run inference on large models with terabytes of data. Our analysis of the technique shows that the runtime is dominated by data transfers of the weights, leading to a low utilization of both the GPU and the CPU. In this paper, we increase the throughput and decrease the total latency of state-of-the-art frameworks by including the CPU as a compute device and overlapping computations on the CPU with GPU data transfers. Our work shows a promising improvement of around 40% in throughput and total latency, with potential room for further improvements.

Background: Recent statistics reveal that 56% of software attacks are caused by insider negligence and 26% are caused by malicious insiders. They also show that 67% of organizations experience at least 21 incidents per year. Most of these incidents require significant time and effort to contain them. In this re-gard, ensuring compliance with corporate policies, regulations, and industry best practices is paramount.Purpose: This study investigates software compliance requirements, factors, and policies together with the challenges they address. By taking a wider perspective, this study aims at bringing an understanding of existing research foci, evolving issues, and research directions.Method: The study uses a systematic literature review and keyword analysis, to identify relevant studies that address the derived research questions. Considering scholarly articles published in the last decade, 4,772 results were retrieved and checked through an initial screening. A thorough screening is then con-ducted to further reduce the results to 77 primary articles.Findings: The requirement on security of end users is gaining more attention. There is an emphasis on the gap between domain and compliance experts on the one side and software engineers on the other side. The review also identified 55 factors (and their underlying theories) that impact behavioral com-pliance with a majority of them focusing on individuals. Our results also list nineteen policies and com-pliance challenges they address. No distinction is found between open-source and proprietary software among the reviewed studies. The most mentioned policies are security education, training, and awareness (SETA), compliance automation, and organizational climate. The evolving topics in the field are: theory of workarounds, compliance and privacy by design, policy as code, security stress, and home-office users.Implications: The review provides 9 recommendations, comprising practical implications for decision makers, theoretical implications for future research, and potential enhancement of the underlying the-ories.(c) 2022 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY license ( http://creativecommons.org/licenses/by/4.0/ )

This manuscript presents the results of the "A View Synthesis Challenge for Humans Heads (VSCHH)", which was part of the ICCV 2023 workshops. This paper describes the competition setup and provides details on replicating our initial baseline, TensoRF. Additionally, we provide a summary of the participants’ methods and their results in our benchmark table. The challenge aimed to synthesize novel camera views of human heads using a given set of sparse training view images. The proposed solutions of the participants were evaluated and ranked based on objective fidelity metrics, such as PSNR and SSIM, computed against unseen validation and test sets. In the supplementary material, we detailed the methods used by all participants in the VSCHH challenge, which opened on May 15th, 2023, and concluded on July 24th, 2023.

In the quest for higher accuracy, large deep neural networks (DNNs) have grown significantly over the past few years. Training and executing large networks with trillions of parameters requires high-end hardware that is expensive to own or rent. A more economical alternative is to distribute the workload to several less powerful but cheaper machines. To devise an efficient workload division, existing parallelization strategies require users to posses intricate knowledge of the model, available hardware, and algorithms. In this paper, we present BBGraP, a device- and model-agnostic black-box partitioner that computes efficient parallelization plans for deep learning inference. For a given network and hardware configuration, BBGraP generates a data-parallel execution plan for each machine. The initial workload partition is optimized by eliminating redundant operations, data transfers, and synchronization points. As a proof-of-concept, BBGraP is applied to a set comprising three distributed nodes and achieves a 30% reduced latency compared to a single node.

Recent neural accelerators often comprise multiple neural processing units (NPUs) with shared cache and memory. The regular schedules of state-of-the-art scheduling techniques miss important opportunities for memory reuse. This paper presents Flexer, an out-of-order (OoO) scheduler that maximizes instruction-level parallelism and data reuse on such multi-NPU systems. Flexer employs a list scheduling algorithm to dynamically schedule the tiled workload to all NPUs. To cope with the irregular data access patterns of OoO schedules, several heuristics help maximize data reuse by considering the availability of data tiles at different levels in the memory hierarchy. Evaluated with several neural networks on 2 to 4-core multi-NPUs, Flexer achieves a speedup of up to 2.2x and a 1.2-fold reduction in data transfers for individual layers compared to the best static execution order.

The era of personal resources being sufficient for enterprise big data computations has passed. As computations are executed in the cloud, small policy changes of cloud operators may cause considerable changes in operational costs. Carefully choosing the amount of resources for a given application is thus of great importance. This, however, requires a priori knowledge of the application's performance under different configurations. Creating a performance prediction model needs to account for the heterogeneity of resources and the diversity in application workflows. Previous approaches for heterogeneous environments consider a black-box representation of the application which results in single-purpose models. This paper addresses the problem with two gray-box prediction models using linear programming (LP) and mixed-integer linear programming (MILP). Given a set of available resources, the models consider Apache Spark applications and their Directed Acyclic Graph (DAG) of workflow running on top of a Hadoop-YARN cluster. We then propose a configuration recommendation algorithm to optimize the cost-performance trade-offs when renting machine instances. The accuracy of the proposed models is evaluated with real-world executions of several representative applications on the Wikipedia dataset and the TPC-DS benchmark. The average error of only 3.28% for the proposed prediction models demonstrates the practicality of the proposed approach in handling cost-performance trade-offs.

Price and quality level of products are two important decisions of any business. This paper provides equilibrium solutions for these decisions of two players for a cybersecurity ecosystem, including a solution provider and an information provider. We assume that end users join a cybersecurity ecosystem based on the prices and qualities of the solution provider's and information provider's products; so that, the increasing of the prices or/and decreasing the qualities will reduce the number of end-users of that ecosystem. Also, it is assumed that there is an exogenous information-sharing program under which the effect of quality of the information provider's product on the cybersecurity level depends on the level of information-sharing. Under this information-sharing program, the equilibrium solutions of prices and qualities of these two players are given. Also, some results and insights are given with a numerical example.

Companies depend on mining data to grow their business more than ever. To achieve optimal performance of Big Data analytics workloads, a careful configuration of the cluster and the employed software framework is required. The lack of flexible and accurate performance models, however, render this a challenging task. This article fills this gap by presenting accurate performance prediction models based on Stochastic Activity Networks (SANs). In contrast to existing work, the presented models consider multiple work queues, a critical feature to achieve high accuracy in realistic usage scenarios. We first introduce a monolithic analytical model for a multi-queue YARN cluster running DAG-based Big Data applications that models each queue individually. To overcome the limited scalability of the monolithic model, we then present a fixed-point model that iteratively computes the throughput of a single queue with respect to the rest of the system until a fixed-point is reached. The models are evaluated on a real-world cluster running the widely-used Apache Spark framework and the YARN scheduler. Experiments with the common transaction-based TPC-DS benchmark show that the proposed models achieve an average error of only $5.6\%$ in predicting the execution time of the Spark jobs. The presented models enable businesses to optimize their cluster configuration for a given workload and thus to reduce their expenses and minimize service level agreement (SLA) violations. Makespan minimization and per-stage analysis are examined as representative efforts to further assess the applicability of our proposition.

Non-uniform memory access (NUMA) architectures exhibit variable memory access latencies that depend on the issuing core and the accessed memory location. To minimize an application's memory access time, the accessed data should be kept as close to the computation as possible. An promising strategy is to deploy groups of threads that access the same data on neighboring cores and close to the accessed data. This not only minimizes remote memory accesses latency but also reduces the amount of accessed cache lines and the traffic incurred by the cache coherence protocol; however, finding and maintaining a good thread group allocation is difficult. This paper presents a novel at-runtime technique that improves application performance through better data locality without prior profiling runs. The presented technique accurately detects accessed memory sections through low-overhead sampling. Sections that are frequently accessed on a remote node are migrated to the local memory node. Migration of unused data such as data streams is avoided by only copying sections that are expected to yield a positive net gain.

Recent desktop and mobile processors often integrate CPU and GPU onto the same die. The limited memory bandwidth of these integrated architectures can negatively affect the performance of data-parallel workloads when all computational resources are active. The combination of active CPU and GPU cores achieving the maximum performance depends on a workload's characteristics, making manual tuning a time-consuming task. Dopia is a fully automated framework that improves the performance of data-parallel workloads by adjusting the Degree Of Parallelism on Integrated Architectures. Dopia transparently analyzes and rewrites OpenCL kernels before executing them with the number of CPU and GPU cores expected to yield the best performance. Evaluated on AMD and Intel integrated processors, Dopia achieves 84% of the maximum performance attainable by an oracle.