The emerging mission-critical applications such as automated driving and remote robotics require mobile wireless networks to provide low latency and high reliability. This paper focuses on the design of coded multipath transmission in wireless networks, where both queue state information and channel state information are considered to reduce delay and packet loss. The latency-sensitive premium traffic is mission-critical, and each traffic packet is required to reach the user via multiple edge servers before a tight deadline. To reduce the packet loss due to queue blocking or channel fading, we apply MDS coding to the multipath transmission at the cost of slightly increasing the communication overhead. For homogeneous systems, we derive the optimal code parameter under the assumption of independent traffic at multiple paths. For heterogeneous systems, we propose greedy and greedy+ algorithms for load allocation. Simulation results show that with traffic dependency at multiple paths our proposed greedy+ algorithm outperforms equal allocation in most cases.

In the deployment of distributed gradient descent algorithms in a network, the computational time and response time of a worker node are affected by various factors, such as processor speed, memory, network delay, and congestion. Straggler feedback delays can severely set back the effectiveness of distributed learning. To leverage the advantage of parallel processing in model training, gradient coding is designed to mitigate the effect of stragglers. This paper investigates gradient coding for heterogeneous workers with varying computational capabilities. We formulate the problem of approximating the gradient vector by minimizing the average error of the recovered gradient vector and propose a solution that leverages fractional repetition codes for data assignment to mitigate the impact of stragglers.

This paper investigates the allocation of coded computing tasks in edge networks with uncertain conditions. Specifically, it focuses on scenarios where the computing speed-related parameters of edge devices are heterogeneous and unknown. The challenge lies in mitigating the impact of stragglers, which involves identifying fast workers and distributing a reasonable workload among them. In this paper, we put the fast worker identification into a multi-armed bandit (MAB) framework and propose a Thompson sampling (TS)-based approach to tackle it. Then, our approach leverages the heterogeneity of computing speeds by formulating the task allocation problem to minimize the expected computing delay. We derive a lower bound for the delay and prove that our proposed algorithm minimizes this bound. Importantly, the time complexity of our algorithm is independent of the number of tasks to be assigned, which is typically large in practical scenarios. When our scheme is applied to solve the linear regression problem, simulation results show that it reduces the computing delay of a state-of-the-art method by more than 15% in two different scenarios.

This paper investigates coded distributed matrix-vector multiplications in the context of eavesdropping, denial-of-service attacks, and computational accuracy. It is demonstrated that the coding problem can be reduced to analog secret sharing. The first analog zigzag-decodable secret-sharing scheme is introduced, and its mutual information security is derived. The encoding and decoding algorithms are shown to be numerically stable, relying solely on shift and addition operations, in contrast to existing methods that require matrix inversions or the solution of linear systems. The proposed scheme achieves comparable security levels while exhibiting a significantly smaller relative computation error than the analog Shamir’s secret-sharing scheme. Additionally, it is shown to outperform the analog Shamir’s scheme in model training accuracy when applied to linear regression and logistic regression problems.

This letter studies online workload allocation for heterogeneous coded edge computing where iterative matrix multiplications are executed. Unlike conventional models assuming known random delay distributions, we consider a realistic scenario where the coordinator only knows that each worker's delay is an affine function of its workload, with random coefficients reflecting communication and computing delays. We formulate a stochastic problem, reduce the dimension to one via estimation, and solve it within a gray-box Bayesian optimization framework. Simulation results show that our approach effectively reduces delay relative to online benchmarks while incurring only a slightly higher delay than offline benchmarks.

Evolutionary algorithms (EAs) have been successful in addressing black-box optimization problems, making them suitable for tackling a wide range of real-world optimization challenges. However, they take much evaluation cost to find the optimal solutions when dealing with problems that involve expensive fitness evaluation functions, which is common in modern systems that involve big data and require hours or even days of simulation. To explore a large search space efficiently, a memetic algorithm can be tailored for a specific EA by combining it with a local search method. This paper presents a general convexity-aware memetic framework (CA-MF) that can be applied to arbitrary EAs. Since the populations of an EA tend to accumulate in the basin of attraction of the landscape, CA-MF employs a partial population from the two most recent generations of the EA to approximate local areas with a computational geometry method based on convex hulls. When a convex-like local landscape is identified, the EA will be switched to local search with a certain probability to accelerate the convergence to a possibly local minimum, saving the computational resources for further global exploration of the EA. To evaluate the performance of CA-MF, it was applied to three different types of EAs, namely CMA-ES, GA, and L-SHADE, and tested on 14 fitness functions as well as real-world applications. The experimental results demonstrate that CA-MF has a generic capacity to enhance the abilities of EAs and outperforms the state-of-the-art memetic framework APrMF. Additionally, the performance analysis of CA-MF is discussed with four local search methods and different parameter configurations. It also shows competitive performance compared with the state of the art method IPOP-CMA-ES.

This letter investigates the workload allocation in heterogeneous distributed computing systems. The optimal energy-delay tradeoff is completely characterized by identifying all the Pareto optimal solutions in a bi-objective optimization problem. For the scenario with a large number of subtasks, the problem is approximated by continuous relaxation, which can be solved with the time complexity of O(N2)$\mathcal {O}(N<^>2)$, where N$N$ is the number of computing devices. Numerical studies show the effectiveness of our approaches and demonstrate the energy-delay tradeoff in distributed computing systems.

This letter examines the decentralized stochastic gradient descent algorithm for federated learning over a wireless ring network, where each device connects to its $2n$ adjacent devices, termed n-tier coverage. Given this topology, the consensus coefficients, or mixing matrix, can be optimized via semidefinite programming (SDP). By employing network coding, the learning topology can be densified without additional communication costs, introducing linear constraints to the uncoded problem. The joint design of the mixing matrix and coding parameters is also formulated as an SDP problem, allowing for efficient determination. Numerical results for linear regression and image classification (using MNIST and CIFAR-10 datasets) demonstrate that our SDP-based network coding approach significantly accelerates convergence in decentralized federated learning under a variation of the pathological non-IID data distribution.

With the development of wireless computing devices, extending distributed computing to wireless networks deserves a closer look. This paper considers distributed computing over unreliable and insecure device-to-device (D2D) networks, in which each device is not always available to perform computation. The process of distributed devices exchanging calculated results with each other is vulnerable to eavesdropping in wireless environments. To handle the unreliable devices, we adopt repetition codes to build a novel system that supports general computations, called ρ-replication system, where each device has ρ - 1 replicas with duplicate data. A coded computation scheme for the ρ-replication system is proposed, which not only achieves the minimum communication load of the system but also ensures weak security of wireless transmissions during data exchange. Furthermore, the replication nature of the system can be exploited for beamforming transmissions, naturally leading to the idea of energy optimization. Simulation results show that increasing ρ does not necessarily improve energy efficiency, as the benefit of increased beamforming gain may be outweighed by the drawback of heavier communication load.

This paper studies distributed storage for protecting the confidentiality of partial data in the presence of storage node failures. It is required that not only the original data can be reconstructed from the remaining surviving nodes, but also the data lost by a failed node can be repaired from as few nodes as possible. The minimum number of surviving nodes required to repair a failed node is called the repair degree. Inspired by the zigzag-decodable secret sharing scheme, we propose a new shift-add secret sharing scheme based on the XOR and bitwise-shift operations, in which confidential data is protected by using random keys generated from non-confidential data. The reliability and repairability of the proposed scheme are measured by the message loss probability and the maximum repair degree among all nodes, respectively, and then compared with three benchmark schemes. In contrast to conventional zigzag-decodable codes, the special structure of our proposed scheme allows the design of fast parallel algorithms for modern devices with multi-core processors, which have a linear speedup in decoding time compared with various versions of serial zigzag decoding. Experiments are implemented on a multi-core computer, and the empirical results on decoding time are consistent with our theoretical observations.

Pure Random Orthogonal Search (PROS) is a parameterless evolutionary algorithm (EA) that has shown superior performance when compared to many existing EAs on well-known benchmark functions with limited search budgets. Its implementation simplicity, computational efficiency, and lack of hyperparameters make it attractive to both researchers and practitioners. However, PROS can be inefficient when the error requirement becomes stringent. In this paper, we propose an extension to PROS, called Pure Random Orthogonal Search with Crossover (PROS-C), which aims to improve the convergence rate of PROS while maintaining its simplicity. We analyze the performance of PROS-C on a class of functions that are monotonically increasing in each single dimension. Our numerical experiments demonstrate that, with the addition of a simple crossover operation, PROS-C consistently and significantly reduces the errors of the obtained solutions on a wide range of benchmark functions. Moreover, PROS-C converges faster than Genetic Algorithms (GA) on benchmark functions when the search budget is tight. The results suggest that PROS-C is a promising algorithm for optimization problems that require high computational efficiency and with a limited search budget.

Surrogate models are commonly employed to reduce computational expenses when dealing with expensive objective optimization problems. This paper introduces a hybrid approach that combines the global exploration capabilities of the Covariance Matrix Adaption Evolution Strategy (CMA-ES) algorithm with the localized search strategy of the Broyden-Fletcher-Goldfarb-Shanno (BFGS) method, incorporating a new non-parametric surrogate model derived from the computational geometry structure of a convex hull in multiple dimensions. After running CMA-ES for some time, the populations of the two most recent iterations are collected to construct the convex approximation of the actual problem landscape. Since the population tends to converge in a landscape basin, the local landscape can be simulated using the constructed convex surrogate model. The accuracy of the convex surrogate model is subsequently validated by the historical solutions and used to determine the probability of switching to a local search. Given BFGS's superior performance in handling unimodal optimization problems, this hybrid approach demonstrates its potential to accelerate the convergence process in finding the global optimum. The experiment is conducted on test functions of the BBOB benchmark with a small evaluation budget, and the results after applying Mann-Whitney U tests confirm the superiority of this method compared to CMA-ES and another global-local framework APrMF.

This paper considers the design of heterogeneous multi-cloud systems for big data storage and computing in the presence of cloud collusion and failures. A fundamental concept of such a system is the secrecy capacity, which represents the maximum amount of information that can be stored for each unit of storage space under the requirements of secure distributed computing. A capacity-achieving code is designed for matrix multiplication, a computing subroutine widely used in machine learning applications. The code allows fast parallel decoding and unequal data allocation in the clouds. Such a flexibility leads naturally to the idea of optimizing data allocation to minimize the computing time. Given any feasible storage budget, the optimal solution is derived, characterizing explicitly the fundamental tradeoff between storage and computing. Furthermore, it is shown via majorization theory that the whole tradeoff curve improves if the cloud computing rates are more even. Experiments on Amazon EC2 clusters are conducted, corroborating our theoretical observations and the negligibility of decoding overhead.

To leverage the computing power in an edge network, one can divide a machine learning task into several subtasks and assign the subtasks to several computing devices to complete. Under master-worker architecture, the master divides and distributes the data to several workers. In each iteration, the master asks the workers to compute some function of the local data stored in the workers. For example, in gradient-based learning, this function can be the partial gradient function. Since the workers have different computing resources, the speed of the distributed learning is hindered by some workers with long latency, called the stragglers. Gradient coding solves the problem of stragglers by allowing the master to recover the desired feedback information in the presence of s stragglers. If the total number of stragglers is n, the master can just wait for the n−s fastest workers. In this paper we consider the problem of data allocation so that the gradient vector can be approximated obtained by the master node with small error. A block repetition scheme is proved to be the optimal data allocation scheme if we want to minimize the average recovery error.

Federated learning is emerging as a new paradigm for joint training of machine learning models across multiple distributed devices. In contrast to many existing works that require a central server to facilitate the exchange of local parameters with a star topology, this work considers fully decentralized operations over a wireless ring network. By increasing the radio coverage of each device, the convergence time, as indicated by the second largest singular value of the weighted adjacency matrix, can be shortened, but the mutual interference will be increased causing larger communication delay. The tradeoff between learning and communication delays is characterized by mathematical analysis of the singular-value gap and by numerical experiments on a linear regression problem. By carefully designing the consensus coefficients of the learning algorithm, a network coding scheme is crafted to improve the entire tradeoff curve without consuming more radio resources. It points to a new direction of using network coding to speed up wireless decentralized learning.

A computing task can be distributed in an edge network and offloaded to multiple edge devices, called workers, to expedite the processing. The computing speeds of the workers, however, are usually unknown or time-varying. To identify the fast workers, a Bayesian approach based on Thompson sampling is used. The estimation of the computing speeds of the workers is formulated as a multi-armed bandit problem. While existing schemes allocate the same amount of computation work to each selected worker, this paper exploits the heterogeneous computing speeds of the workers and formulates the task allocation problem with the objective of minimizing the overall computing delay. A lower bound for the delay is obtained and is proved to be minimized by a greedy algorithm. Simulation results show that our scheme outperforms other benchmarks.

In this paper, the behavior and performance of Pure Random Orthogonal Search (PROS), a parameter-free evolutionary algorithm (EA) that outperforms many existing EAs on the well-known benchmark functions with finite-time budget, are analyzed. The sufficient conditions to converge to the global optimum are also determined. In addition, we propose two modifications to PROS, namely Triangular-Distributed Random Orthogonal Search (TROS) and Quadratic-Distributed Random Orthogonal Search (QROS). With our local search mechanism, both modified algorithms improve the convergence rates and the errors of the obtained solutions significantly on the benchmark functions while preserving the advantages of PROS: parameterless, excellent computational efficiency, ease of applying to all kinds of applications, and high performance with finite-time search budget. The experimental results show that both TROS and QROS are competitive in comparison to several classic metaheuristic optimization algorithms.

Maximum completion time is a key performance metric for future wireless applications that have stringent latency requirements. This letter studies its optimization in uplink multi-subcarrier non-orthogonal multiple access (NOMA) networks. The joint optimization problem of power allocation, user pairing and scheduling is analyzed and proved to be NP-hard. To tackle the problem, the power allocation subproblem for a single NOMA pair is first derived. Next, for the joint user pairing and scheduling subproblem, an optimal algorithm for a special case is constructed and a suboptimal algorithm for the general case is designed. Our proposed scheme is applicable to data aggregation in wireless federated learning networks, and simulation results validate its superior performance.

Recently, over-the-air computation is considered an efficient scheme for enormous data transmission in distributed learning and computing systems. Its performance is limited by the aggregation errors, which may be caused by noise, channel fading, and insufficient device power budgets. Inspired by gradient coding, this paper considers to leverage the computing abilities of the edge devices to reap a diversity gain and alleviate the effects of inadequate transmit power. The edge server divides the whole dataset into subsets and distributes them to edge devices by some data assignment scheme. The edge devices send the computation results simultaneously back to the edge server by over-the-air transmission. This paper jointly optimizes the data assignment and power allocation problems in over-the-air distributed learning systems to minimize the mean square error (MSE) of the aggregation data. Given the data assignment scheme, the power allocation problem is solved optimally by block coordinate descent (BCD) and grid search. Besides, some optimality conditions for data assignment are proved. Accordingly, a heuristic data assignment scheme is proposed. Numerical results show our proposed scheme outperforms existing works in terms of MSE and learning metrics.