Modern production data processing and machine learning pipelines on the cloud are critical components for many cloud-based companies. These pipelines are typically composed of complex workflows represented by directed acyclic graphs (DAGs). Cloud environments are attractive to these workflows due to the wide range of choice with heterogeneous instances and prices that can provide the flexibility for different cost-performance needs. However, this flexibility also leads to the complexity of selecting the right resource configuration (e.g., instance type, resource demands) for each task in the DAG, while simultaneously scheduling the tasks with the selected resources to reach the optimal end-to-end performance and cost. These two decisions are often codependent resulting in an NP-hard scheduling optimization bottleneck. Existing solutions only focus solely on either problem and ignore the co-effect on the end-to-end optimum. We propose AGORA, a scheduler that considers both task-level resource allocation and execution for DAG workflows as a whole in heterogeneous cloud environments. AGORA first (1) studies the characteristics of the tasks from prior runs and gives predictions on resource configurations, and (2) automatically finds the best configuration with its corresponding schedules for the entire workflow with a cost-performance objective. We evaluate AGORA in a heterogeneous Amazon Web Services (AWS) cloud environment with multi-tenant workflows served by Airflow and demonstrate a performance improvement up to 45% and cost reduction up to 77% compared to state-of-the-art schedulers. In addition, we apply AGORA to a real-world production trace from Alibaba and show cost reduction of 65% and DAG completion time reduction of 57%.

This special issue of the Swarm Intelligence journal is dedicated to the publication of extended versions of the best papers presented atANTS 2014, Ninth International Conference on Swarm Intelligence, which took place in Brussels on September 10–12, 2014. The ANTS series of conferences has taken place at the Universite Libre de Bruxelles, Brussels, Belgium, every other year since 1998. As in 2010 and in 2012 (for the seventh and eighth editions of the conference), the authors of the contributions accepted as full papers at the conferencewere invited to submit an extended version of their work for possible inclusion in this special issue.

The benchmark functions and some of the algorithms proposed for the special session on real parameter optimization of the 2005 IEEE Congress on Evolutionary Computation (CEC'05) have played and still play an important role in the assessment of the state of the art in continuous optimization. In this article, we show that if bound constraints are not enforced for the final reported solutions, state-of-the-art algorithms produce infeasible best candidate solutions for the majority of functions of the IEEE CEC'05 benchmark function suite. This occurs even though the optima of the CEC'05 functions are within the specified bounds. This phenomenon has important implications on algorithm comparisons, and therefore on algorithm designs. This article's goal is to draw the attention of the community to the fact that some authors might have drawn wrong conclusions from experiments using the CEC'05 problems.

In this paper, we present a continuous-time binary consensus protocol whereby entities connected via a directed ring topology solve the one-dimensional density classification problem. In our model, the participating entities behave as non-ideal relays, that is, they have memory of the trajectory of an internal state variable, which gives them hysteretic properties. We show that this feature is necessary for the system to reach consensus on the state shared by the initial majority. The connections between this protocol and collective decisionmaking mechanisms in swarm intelligence systems are also discussed. Introduction The density classification problem (also known as the majority problem), consists in classifying finite linear binary strings according to whether they have a majority of 0’s or 1’s. This problem has been the subject of numerous studies in the cellular automata literature (see, e.g., Mitchell et al. (1994); Land and Belew (1995); Fukś (1997, 2002); AlonsoSanz and Bull (2009); Fates (2013)) because it has a simple formulation and illustrates very well the idea of “emergent computation” since the cells interact only locally and do not have access to the global structure they are trying to classify. In a cellular automata setting, the density classification problem translates into finding evolution rules for cells that make the automata be all equal to 0 if initially there were more than half of the cells in a 0 state, or 1 if initially there were more than half of the cells in a 1 state. The idea of emergent computation is also present in another computational paradigm called swarm intelligence (Bonabeau et al., 1999; Dorigo and Birattari, 2007). In this paradigm, relatively simple agents locally interact with one another and with their environment to produce self-organized spatio-temporal patterns that represent solutions to problems that no individual agent could solve on its own (e.g., finding shortest paths (Goss et al., 1989), sorting (Deneubourg et al., 1990), or constructing nests (Grasse, 1959)). As we explain in the next section, the density classification problem is relevant in swarm intelligence because some of the methods that solve it may be used as collective decision-making mechanisms for swarms. In this paper, we further explore the connection between cellular automata and swarm intelligence by presenting and analyzing a consensus protocol1 on networks of agents derived from previous work on collective decision-making in swarms (Montes de Oca et al., 2012). Our consensus protocol may be seen as encoding evolution rules for continuous-time cellular automata with memory. It may also be thought of as a model of social influence in a group whose members observe the actions performed by other individuals, increasing, as a result, their tendency to perform the observed actions. We are interested in this type of mechanisms because we want to eventually endow swarms of robots or agents with collective-decision mechanisms that are robust, flexible and effective in real environments. Our approach is backed by recent studies on social learning (Rendell et al., 2010), that support the idea that learning from the observation of others’ actions is a mechanism whereby individuals indirectly probe the environment. In a swarm, which is typically composed of many individuals, using the behavior of others as a guide provides each individual with potentially many indirect channels for obtaining information about their environment, and thereby increasing the amount of information they use to make decisions. The key finding presented in this paper is that if agents influence each other as if arranged in a unidirectional ring, and each agent integrates over time information coming to it from its neighbor using a mechanism akin to exponential smoothing (Gardner Jr., 2006), then symmetric blocks of 0’s or 1’s propagate through the network indefinitely. Moreover, we provide evidence that when the symmetry of these blocks is broken (that is, there is a majority of 0’s or 1’s), then the information wave propagates for a finite amount of time and eventually dies out, which translates into the population of agents reaching a consensus. Finally, the state on which the We use the term protocol to comply with literature tradition in communication networks and where interaction rules are called protocols. See, for example, (Mesbahi and Egerstedt, 2010). ALIFE 14: Proceedings of the Fourteenth International Conference on the Synthesis and Simulation of Living Systems population reaches a consensus corresponds to that of the initial majority. In other words, we provide evidence that the proposed continuous-time consensus protocol with hysteretic units solves the density classification problem. Ongoing work is aimed at analytically determining how much time is needed for the system to converge. Collective Decision-Making in Swarms In (Montes de Oca et al., 2012), we proposed a social influence model whose dynamics can be used as a collective decision-making mechanism for swarms of robots that need to collectively choose the most efficient of two alternative actions (henceforth referred to as Decision-Making Model or DM model). In the DM model, each of a set of n agents can be in one of two states (represented with a binary variable Xi ∈ {0, 1}, with i = 1, 2, . . . , n). In applications of the DM model, an agent’s state can represent, for example, a robot’s preferred action or current belief of the state of an environmental variable. The DM model is a discrete-time model where at each time step t of the system’s evolution, an agent imight be able to observe the state of another random agent j 6= i. When agent i observes the state of another agent j, the observing agent i updates an internal real-valued variable Si, which we call tendency, as follows: S t+1 i = (1− α)S t i + αX t j , (1) where 0 ≤ α ≤ 1 determines the relative weight given to the agent’s latest observation (X j) and the agent’s accumulated experience (S i ). After updating its tendency, an agent updates its state as follows: X i =  1, if S t+1 i ≥ λ 0, if S t+1 i ≤ μ X i , if μ < S t+1 i < λ , (2) where μ+ λ = 1 (the reason for this constraint will become apparent later). Eq. 2 implements a sort of dynamic memory that allows the agent to integrate its observations over time. By properly choosing values for the parameters α, μ, λ, and the initial conditions X i and S 0 i , one can control the imitation behavior of agent i. While in principle, each agent may have different values for its parameters, in the DM model, α, λ and μ, are constant and common to all agents. An example of the behavior of an individual agent in the DM model is shown in Figure 1. In the DM model the population is reshuffled randomly so that each individual observes a different agent at each time step. Additionally, one single agent may potentially influence more than one other agent in the group. The DM model’s collective dynamics make the population reach a consensus on the state that at time step 0 is shared by most (that is, the majority) of the population. In (Montes de Oca 0 50 100 150 200 0 .0 0 .2 0 .4 0 .6 0 .8 1 .0 Observation Number T e n d e n c y /S ta te Figure 1: Single agent behavior in the DM model. Starting with an initialization of S = 0.5 and X = 0, an agent observes a stream of state values plotted as dots with values 0 or 1. The black line shows the evolution of the agent’s tendency and the blue line shows the evolution of the agent’s state. In this simulation, α = 0.2, λ = 0.75 and μ = 0.25 (shown as dotted lines). et al., 2012), and (Montes de Oca et al., 2011), we show how this behavior can be used for optimal collective decisionmaking in robot swarms. The DM model may be seen as a method to solve the density classification problem if its definition is relaxed. In particular, if the agents (cells) are allowed to be reshuffled, then the DM may solve it. In this paper, we explore the question of whether it is possible to solve the original density classification problem with a variation of the DM model that does not require reshuffling. In the following sections, we present such a variation as well as theoretical and experimental results that make us believe that the question can be answered positively. Continuous-Time Consensus Protocol with Hysteretic Units The protocol that we propose in this paper, henceforth referred to as Consensus with Hysteresis or CH, is in its basic form the continuous-time equivalent of Eq. 1. However, in the CH protocol, the communication topology of the population does not change over time and each agent influences exactly one other agent. In the remainder of this paper, we assume that individuals are arranged in a directed ring topology with an agent i influenced by agent j, where j = i+1 or j = i− 1 (agents “look” to their right or left, respectively). ALIFE 14: Proceedings of the Fourteenth International Conference on the Synthesis and Simulation of Living Systems

In this article, we propose UACOR, a unified ant colony optimization (ACO) algorithm for continuous optimization. UACOR includes algorithmic components from ACO(R), DACO(R) and IACO(R)-LS, three ACO algorithms for continuous optimization that have been proposed previously. Thus, it can be used to instantiate each of these three earlier algorithms; in addition, from UACOR we can also generate new continuous ACO algorithms that have not been considered before in the literature. In fact, UACOR allows the usage of automatic algorithm configuration techniques to automatically derive new ACO algorithms. To show the benefits of UACOR's flexibility, we automatically configure two new ACO algorithms, UACOR-s and UACOR-c, and evaluate them on two sets of benchmark functions from a recent special issue of the Soft Computing (SOCO) journal and the IEEE 2005 Congress on Evolutionary Computation (CEC'05), respectively. We show that UACOR-s is competitive with the best of the 19 algorithms benchmarked on the SOCO benchmark set and that UACOR-c performs superior to IPOP-CMA-ES and statistically significantly better than five other algorithms benchmarked on the CEC'05 set. These results show the high potential ACO algorithms have for continuous optimization and suggest that automatic algorithm configuration is a viable approach for designing state-of-the-art continuous optimizers. (C) 2013 Elsevier B.V. All rights reserved.

In this paper, we introduce ACO(MV) : an ant colony optimization (ACO) algorithm that extends the ACO(R) algorithm for continuous optimization to tackle mixed-variable optimization problems. In ACO(MV), the decision variables of an optimization problem can be explicitly declared as continuous, ordinal, or categorical, which allows the algorithm to treat them adequately. ACO(MV) includes three solution generation mechanisms: a continuous optimization mechanism (ACO(R)), a continuous relaxation mechanism (ACO(MV)-o) for ordinal variables, and a categorical optimization mechanism (ACO(MV)-c) for categorical variables. Together, these mechanisms allow ACO(MV) to tackle mixed-variable optimization problems. We also define a novel procedure to generate artificial, mixed-variable benchmark functions, and we use it to automatically tune ACO(MV)'s parameters. The tuned ACO(MV) is tested on various real-world continuous and mixed-variable engineering optimization problems. Comparisons with results from the literature demonstrate the effectiveness and robustness of ACO(MV) on mixed-variable optimization problems.

Swarm intelligence is the collective problem-solving behavior of groups of animals and artificial agents. Often, swarm intelligence is the result of self-organization, which emerges from the agents’ local interactions with one another and with their environment. Such local interactions can be positive, negative, or neutral. Positive interactions help a swarm of agents solve a problem. Negative interactions are those that block or hinder the agents’ task-performing behavior. Neutral interactions do not affect the swarm’s performance. Reducing the effects of negative interactions is one of the main tasks of a designer of effective swarm intelligence systems. Traditionally, this has been done through the complexification of the behavior and/or the characteristics of the agents that comprise the system, which limits scalability and increases the difficulty of the design task. In collaboration with colleagues, I have proposed a framework, called incremental social learning (ISL), as a means to reduce the effects of negative interactions without complexifying the agents’ behavior or characteristics. In this paper, I describe the ISL framework and three instantiations of it, which demonstrate the framework’s effectiveness. The swarm intelligence systems used as case studies are the particle swarm optimization algorithm, ant colony optimization algorithm for continuous domains, and the artificial bee colony optimization algorithm.

Swarm robotics systems are characterized by decentralized control, limited communication between robots, use of local information, and emergence of global behavior. Such systems have shown their potential for flexibility and robustness [1]-[3]. However, existing swarm robotics systems are by and large still limited to displaying simple proof-of-concept behaviors under laboratory conditions. It is ...

In this paper, we reinterpret the most basic exponential smoothing equation, S t + 1 = (1 − α)S t  + αX t , as a model of social influence. This equation is typically used to estimate the value of a series at time t + 1, denoted by S t + 1, as a convex combination of the current estimate S t and the actual observation of the time series X t . In our work, we interpret the variable S t as an agent’s tendency to adopt the observed behavior or opinion of another agent, which is represented by a binary variable X t . We study the dynamics of the resulting system when the agents’ recently adopted behaviors or opinions do not change for a period of time of stochastic duration, called latency. Latency allows us to model real-life situations such as product adoption, or action execution. When different latencies are associated with the two different behaviors or opinions, a bias is produced. This bias makes all the agents in a population adopt one specific behavior or opinion. We discuss the relevance of this phenomenon in the swarm intelligence field.

Automated algorithm configuration methods have proven to be instrumental in deriving high-performing algorithms and such methods are increasingly often used to configure evolutionary algorithms. One major challenge in devising automatic algorithm configuration techniques is to handle the inherent stochasticity in the configuration problems. This article analyses a post-selection mechanism that can also be used for this task. The central idea of the post-selection mechanism is to generate in a first phase a set of high-quality candidate algorithm configurations and then to select in a second phase from this candidate set the (statistically) best configuration. Our analysis of this mechanism indicates its high potential and suggests that it may be helpful to improve automatic algorithm configuration methods.

In this article, we apply an automatic algorithm configuration tool to improve the performance of the CMA-ES algorithm with increasing population size (iCMA-ES), the best performing algorithm on the CEC'05 benchmark set for continuous function optimization. In particular, we consider a separation between tuning and test sets and, thus, tune iCMA-ES on a different set of functions than the ones of the CEC'05 benchmark set. Our experimental results show that the tuned iCMA-ES improves significantly over the default version of iCMA-ES. Furthermore, we provide some further analyses on the impact of the modified parameter settings on iCMA-ES performance and a comparison with recent results of algorithms that use CMA-ES as a subordinate local search.

In previous work, we introduced a novel swarming interpolation framework and validated its effectiveness on static fields. In this paper, we show that a slightly revised version of this framework is able to track fields that translate, rotate, or expand over time, enabling interpolation of both static and dynamic fields. Our framework can be used to control autonomous mobile sensors into flexible spatial arrangements in order to interpolate values of a field in an unknown region. The key advantage to this framework is that the stable sensor distribution can be chosen to resemble a Chebyshev distribution, which can be optimal for certain ideal geometries.

At an abstract level, memetic algorithms can be seen as a broad class of populationbased stochastic local search (SLS) methods, where a main theme is “exploiting all available knowledge about a problem,” see also Moscato and Cotta [618], page 105. The most wide-spread implementation of this theme is probably that of improving some or all individuals in the population by some local search method. This combination of a population-based, global search and a single-solution local search is a very appealing one. The global search capacity of the evolutionary part of a memetic algorithm takes care of exploration, trying to identify the most promising search space regions; the local search part scrutinizes the surroundings of some initial solution, exploiting it in this way. This idea is not only an appealing one, it is also practically a very successful one. In fact, for a vast majority of combinatorial optimization problems and, as it is also becoming more clear in recent research, also for many continuous optimization problems this combination leads to some of best performing heuristic optimization algorithms.

Reverse engineering is the problem of inferring the structure of a network of interactions between biological variables from a set of observations. In this paper, we propose an optimization algorithm, called MORE, for the reverse engineering of biological networks from time series data. The model inferred by MORE is a sparse system of nonlinear differential equations, complex enough to realistically describe the dynamics of a biological system. MORE tackles separately the discrete component of the problem, the determination of the biological network topology, and the continuous component of the problem, the strength of the interactions. This approach allows us both to enforce system sparsity, by globally constraining the number of edges, and to integrate a priori information about the structure of the underlying interaction network. Experimental results on simulated and real-world networks show that the mixed discrete/continuous optimization approach of MORE significantly outperforms standard continuous optimization and that MORE is competitive with the state of the art in terms of accuracy of the inferred networks.

We modify an artificial bee colony algorithm as follows: we make the population size grow over time and apply local search on strategically selected solutions. The modified algorithm obtains very good results on a set of large-scale continuous optimization benchmark problems. This is not the first time we see that the two aforementioned modifications make an initially non-competitive algorithm obtain state-of-the-art results. In previous work, we have shown that the same modifications substantially improve the performance of particle swarm optimization and ant colony optimization algorithms. Altogether, these results suggest that population growth coupled with local search help obtain high-quality results.

In this paper, we describe a novel swarming framework that guides autonomous mobile sensors into a flexible arrangement to interpolate values of a field in an unknown region. The algorithm is devised so that the sensor distribution will behave like a Chebyshev distribution, which can be optimal for certain ideal geometries. The framework is designed to dynamically adjust to changes in the region of interest, and operates well with very little a priori knowledge of the given region.For comparison, we interpolate a variety of nontrivial fields using a standard swarming algorithm that produces a uniform distribution and our new algorithm. We find that our new algorithm interpolates fields with greater accuracy.

ACO(R) is an ant colony optimization algorithm for continuous domains. In this article, we benchmark ACO(R) on the BBOB noiseless function testbed, and compare its performance to PSO, ABC and GA algorithms from previous BBOB workshops. Our experiment shows that ACO(R) performs better than PSO, ABC and GA on the moderate functions, ill-conditioned functions and multi-modal functions. Among 24 functions, ACO(R) solved 19 in dimension 5, 9 in dimension 20, and 7 across dimensions from 2 to 40. Furthermore, in dimension 5, we present the results of the ACO(R) when it uses variable correlation handling. The latter version is competitive on the five dimensional functions to (1+1)-CMA-ES and BIPOP-CMA-ES.

One of the most popular Ant Colony Optimization (ACO) algorithms for continuous optimization problems is ACOR. In this paper, we propose an incremental ACOR with local search (IACOR-LS) that obtains better results than the original ACOR on a number of continuous optimization problems. We first present a mechanism that improves the search diversification of ACOR. This mechanism consists of a growing solution archive with a special initialization rule applied to entrant solutions. The resulting algorithm, called IACOR, is then hybridized with a local search procedure in order to enhance its search intensification. We experiment with Powell’s conjugate directions set, Powell’s BOBYQA, and Lin-Yu Tseng’s Mtsls1 methods as local search procedures. Automatic parameter tuning results show that IACOR-LS with Mtsls1 (IACOR-Mtsls1) is not only a significant improvement over ACOR, but that it is also competitive with the state-of-the-art algorithms described in a recent special issue of the journal Soft Computing. Further experimentation with IACOR-Mtsls1 on the combined benchmark functions suite from both the special issue of Soft Computing and the IEEE 2005 Congress on Evolutionary Computation (CEC 2005) demonstrates its good performance in continuous optimization.