Community Detection (CD) in weighted social networks is a highly active research field, celebrated for its profound practical implications across a multitude of disciplines. Genetic algorithms (GAs) are frequently explored to tackle CD problems, leveraging their capability to navigate the extensive discrete search space effectively. Throughout the evolutionary process, genetic operators such as crossover and mutation assume pivotal roles in effectively exploring the vast solution space. Nonetheless, prevailing GA-based approaches often ignore crucial topology information, particularly information regarding edge weights, resulting in compromised algorithm performance. In light of this, this paper introduces Edge Information-based GA (EIGA) to effectively solve CD problems in weighted networks. This is achieved specifically through the innovative designs of edgeweight-aware crossover and mutation operators. These novel edge-weight-aware operators improve the extraction of meaningful community structures, advancing knowledge discovery from social networks. Empirical findings demonstrate the superior performance of EIGA over numerous state-of-the-art algorithms across various real-world and synthetic benchmark networks.

Bus timetable optimization is crucial for a bus-centric public transportation system. Existing bus timetable optimization methods typically produce fixed timetables based on historical passenger flows, limiting their ability to meet real-time passenger demand fluctuations. Although dynamic optimization methods based on deep reinforcement learning (DRL) have been explored recently, they often fail to accurately capture real-time passenger demand and prioritize minimizing costs at the expense of service quality. Additionally, their black-box nature limits transparency, potentially introducing redundant state features that hinder the model's ability to capture passenger demands and reduce stability. In this paper, we propose a new eXplainable Reinforcement Learning-based approach for bus Timetable dynamic Optimization (XRL-TO). Specifically, a novel Markov Decision Process (MDP) model is developed to effectively capture passenger demand and balance service quality and operating costs. An Attention-based Deep Q-Network (ADQN) is employed as the agent to process the new state representation. To enhance its transparency, a LIME-based Reinforcement Learning eXplainability method (LRLX) is proposed to analyze the decision-making process of ADQN and systematically refine the state representation, improving both the transparency and stability of XRL-TO. Experimental results on real-world data demonstrate that XRL-TO significantly reduces both operating costs and passengers' average waiting time compared to state-of-the-art algorithms. Furthermore, LRLX substantially enhances the transparency and stability of our approach, making it highly valuable for bus timetable dynamic optimization in practice.

Cost-aware Dynamic Workflow Scheduling (CADWS) is a key challenge in cloud computing, focusing on devising an effective scheduling policy to efficiently schedule dynamically arriving workflow tasks, represented as Directed Acyclic Graphs (DAG), to suitable virtual machines (VMs). Deep reinforcement learning (DRL) has been widely employed for automated scheduling policy design. However, the performance of DRL is heavily influenced by the design of the problem-tailored policy network and is highly sensitive to hyperparameters and the design of reward feedback. Considering the above-mentioned issues, this study proposes a novel DRL method combining Graph Attention Networks-based policy network and Evolution Strategy, referred to as GATES. The contributions of GATES are summarized as follows: (1) GATES can capture the impact of current task scheduling on subsequent tasks by learning the topological relationships between tasks in a DAG. (2) GATES can assess the importance of each VM to the ready task, enabling it to adapt to dynamically changing VM resources. (3) Utilizing Evolution Strategy's robustness, exploratory nature, and tolerance for delayed rewards, GATES achieves stable policy learning in CADWS. Extensive experimental results demonstrate the superiority of the proposed GATES in CADWS, outperforming several state-of-the-art algorithms. The source code is available at: https://github.com/YaShen998/GATES.

Accurate assessment of biochemical compositions in fish products is essential for quality control in the seafood industry and nutritional research. While spectroscopic techniques enable non-destructive analysis, each method has limitations in prediction accuracy and reliability. Multi-modal data fusion offers a promising solution, but developing robust fusion strategies remains challenging due to complex relationships between spectral features and biochemical properties. This paper presents GP-Fusion, a genetic programming-based high-level fusion method that integrates multiple spectroscopic modalities. Unlike conventional approaches, GPFusion evolves interpretable fusion functions to optimize predictions from diverse spectroscopy work-flows. A key innovation is the replicate variance penalty, which enhances prediction consistency across replicate measurements by capturing within-sample variability and mitigating batch effects. Experimental evaluations on three biochemical targets, including Omega-3, Omega-6, and monounsaturated fatty acids, show that GPFusion improves the coefficient of determination by 6.9%, 8.0%, and 2.6%, respectively. Compared with other high-level fusion strategies, GPFusion delivers more stable predictions with lower variance while maintaining competitive accuracy. Additional empirical studies confirms the effectiveness of the replicate variance penalty and reveal critical trade-offs between tree depth and terminal flexibility for evolving compact and interpretable fusion functions.

Community detection, a vital technology for real-world applications, uncovers cohesive node groups (communities) by leveraging both topological and attribute similarities in social networks. However, existing Graph Convolutional Networks (GCNs) trained to maximize modularity often converge to suboptimal solutions. Additionally, directly using human-labeled communities for training can undermine topological cohesiveness by grouping disconnected nodes based solely on node attributes. We address these issues by proposing a novel Topological and Attributive Similarity-based Community detection (TAS-Com) method. TAS-Com introduces a novel loss function that exploits the highly effective and scalable Leiden algorithm to detect community structures with global optimal modularity. Leiden is further utilized to refine human-labeled communities to ensure connectivity within each community, enabling TAS-Com to detect community structures with desirable trade-offs between modularity and compliance with human labels. Experimental results on multiple benchmark networks confirm that TAS-Com can significantly outperform several state-of-the-art algorithms.

Integrated order batching and picker routing (IOBPR) is a complex combinatorial optimization problem in real-world intelligent manufacturing systems. Heuristics are often used for solving such complex scheduling problems. Manually designing scheduling heuristics suffer from two limitations: 1) few problem features can be taken into account and 2) the design process is time consuming. Genetic programming hyper heuristic (GPHH) approaches have been proposed on many scheduling problems to automatically evolve effective heuristics. However, existing GPHH approaches are often problem specific and requires careful design of problem specific terminal sets and evolution operators. The aim of this work is to develop a GPHH approach to evolve heuristics for the IOBPR problem. In particular, we propose a novel terminal set (NT) with three types of terminals, and a GPHH with elitist mutation (GPHH-EM) algorithm. Extensive experiments demonstrate that the heuristics evolved by GPHH-EM can significantly outperform other state-of-the-art competing algorithms designed by human experts. Further analysis indicates that the three types of terminals effectively complement to improve evolved heuristics for decision making. Furthermore, the newly developed elitist mutation operator expedites the evolutionary process for GPHH to find high-quality heuristics.

Dynamic workflow scheduling (DWS) in cloud computing presents substantial challenges due to heterogeneous machine configurations, unpredictable workflow arrivals/patterns, and constantly evolving environments. However, existing research often assumes homogeneous setups and static conditions, limiting flexibility and adaptability in real-world scenarios. In this paper, we propose a novel *Graph assisted Offline-Online Deep Reinforcement Learning* (GOODRL) approach to building an effective and efficient scheduling agent for DWS. Our approach features three key innovations: (1) a *task-specific* graph representation and a *Graph Attention Actor Network* that enable the agent to dynamically assign focused tasks to heterogeneous machines while explicitly considering the future impact of each machine on these tasks; (2) a *system-oriented* graph representation and a *Graph Attention Critic Network* that facilitate efficient processing of new information and understanding its impact on the current state, crucial for managing unpredictable workflow arrivals/patterns in real-time; and (3) an *offline-online* method that utilizes imitation learning for effective offline training and applies gradient control and decoupled high-frequency critic training techniques during online learning to sustain the agent’s robust performance in rapidly changing environments. Experimental results demonstrate that GOODRL significantly outperforms several state-of-the-art algorithms, achieving substantially lower mean flowtime and high adaptability in various online and offline scenarios.

Many machine learning models are susceptible to adversarial attacks, with decision-based black-box attacks representing the most critical threat in real-world applications. These attacks are extremely stealthy, generating adversarial examples using hard labels obtained from the target machine learning model. This is typically realized by optimizing perturbation directions, guided by decision boundaries identified through query-intensive exact search, significantly limiting the attack success rate. This paper introduces a novel approach using the Approximation Decision Boundary (ADB) to efficiently and accurately compare perturbation directions without precisely determining decision boundaries. The effectiveness of our ADB approach (ADBA) hinges on promptly identifying suitable ADB, ensuring reliable differentiation of all perturbation directions. For this purpose, we analyze the probability distribution of decision boundaries, confirming that using the distribution's median value as ADB can effectively distinguish different perturbation directions, giving rise to the development of the ADBA-md algorithm. ADBA-md only requires four queries on average to differentiate any pair of perturbation directions, which is highly query-efficient. Extensive experiments on six well-known image classifiers clearly demonstrate the superiority of ADBA and ADBA-md over multiple state-of-the-art black-box attacks.

Containers have emerged as a prevalent mechanism for deploying software applications within cloud data centers, thereby abstracting many operational details from developers and transferring the responsibility of resource management to cloud service providers. These providers are continually motivated to reduce operational costs by optimizing both the placement of containers and the Virtual Machines (VMs) that host them, as efficient placement directly contributes to reduced energy expenditures. Furthermore, given that energy prices vary both temporally and geographically due to fluctuating power production, demand, and the increasing integration of renewable en-ergy sources, explicit consideration of dynamic, location-specific energy pricing is essential for informed resource allocation and container migration decisions. In this work, we propose a novel container-based cloud resource allocation model that integrates variable energy prices across multiple locations. To address this problem, we introduce an innovative Genetic Programming Hyper-Heuristic (GPHH) algorithm that concurrently evolves three heuristics for container allocation, VM placement, and container migration. A key technical novelty of our approach is the incorporation of newly designed terminals within the GPHH framework that are specifically engineered to capture and utilize the complex dynamic power pricing information. Experimental results demonstrate that our GPHH algorithm offers significant improvements over several state-of-the-art methods, thereby enhancing both cost efficiency and energy optimization in cloud environments.

Effectively scheduling the execution of deadline-constrained IoT workflows in Fog-Cloud computing is an im-portant practical problem. The execution of IoT workflows must be carefully controlled, especially in applications like IoT health, where missed deadlines can be life-threatening. Existing approaches, including heuristic and meta-heuristic algorithms, have limited capabilities of handling deadline constraints while ensuring efficient execution of workflows. In this paper, we develop a Genetic Algorithm (GA) with a novel repair method to properly allocate workflow tasks to IoT devices and Fog/Cloud servers in order to meet deadlines while minimizing the resource costs. To effectively handle deadline constraints, our repair method first identifies infeasible solutions evolved by GA that can be potentially repaired. It then carries out root cause analysis to determine the primary reasons for deadline violation, and finally reallocate critical tasks to resolve violated deadlines. In this way, the risks of violating deadline constraints can be substantially reduced. On a wide range of problem instances commonly studied in literature, our experiments show that the new algorithm can clearly outperform multiple recently developed methods since it enjoys better chance of finding solutions that satisfy deadline constraints and can notably reduce the resource cost simultaneously.

In this paper, we propose a novel policy gradient algorithm for deep reinforcement learning. Unlike previous approaches, we focus on leveraging the Hessian trace information in the policy parametric space to enhance the performance of trained policy networks. Specifically, we introduce a metric tensor field that transforms the policy parametric space into a general Riemannian manifold. We further develop mathematical tools, deep learning algorithms, and metric tensor deep neural networks (DNNs) to learn a desirable metric tensor field, with the aim to achieve close-to-zero divergence on the policy gradient vector field of the Riemannian manifold. As an important regularization mechanism, zero divergence nullifies the principal differential components of the loss function used for training policy networks. It is expected to improve the effectiveness and sample efficiency of the policy network training process. Experimental results on multiple benchmark reinforcement learning problems demonstrate the advantages of our metric tensor regularized algorithms over the non-regularized counterparts. Moreover, our empirical analysis reveals that the trained metric tensor DNN can effectively reduce the absolute divergence towards zero on the Riemannian manifold.

In the domain of Cloud computing, Fog computing is integrated with the Cloud to offer a balanced approach that combines Cloud's scalability with Fog's low latency, enabling efficient software application deployment. However, many current studies overlook the unpredictability of future user requests, such as assuming all requests are known beforehand. User requests often arrive dynamically and may have different quality of service (QoS) preferences. Therefore we need effective methods to handle dynamic application deployment with multiple objectives. This paper tackles this gap by modeling a multi-objective application deployment problem that considers dynamically arriving users' requests on application deployment in a Cloud-Fog environment. We further introduce a multi-objective Genetic Programming Hyper-Heuristic based approach to automatically generate a set of deployment rules that can be chosen according to users' QoS preferences. These rules are generated with different trade-offs of two optimization objectives, i.e., minimizing cost and latency, which can be used for deploying applications dynamically. Our experimental evaluation using real-world data demonstrates that our GPHH approach can generate effective heuristics for deploying applications in an integrated Cloud-Fog environment.

Genetic Programming Hyper-heuristics (GPHH) have been successfully used to evolve scheduling rules for Dynamic Workflow Scheduling (DWS) as well as other challenging combinatorial optimization problems. The method of sampling training instances has a significant impact on the generalization ability of GPHH, yet they are rarely addressed in existing research. This article aims to fill this gap by proposing a GPHH algorithm with a sampling strategy to thoroughly investigate the impact of six instance sampling strategies on algorithmic generalization, including one rotation strategy, three mini-batch strategies, and two hybrid strategies. Experiments across four scenarios with varying settings reveal that: (1) mini-batch with random sampling can outperform rotation in generalizing to unseen workflow scheduling problems under the same computational cost; (2) employing a hybrid strategy that combines rotation and mini-batch further enhances the generalization ability of GPHH; and (3) mini-batch and hybrid strategies can effectively enable heuristics trained on small-scale training instances generalizing well to large-scale unseen ones. These findings highlight the potential of mini-batch strategies in GPHH, offering improved generalization performance while maintaining diversity and suggesting promising avenues for further exploration in GPHH domains.

Containers are a popular way of deploying software in cloud data centers. Containers are allocated to Virtual machines (VMs) which are allocated to Physical machines (PMs) within the data center. Since the resources required by containers often do not match those of VMs, where to allocate them must be decided. A poor solution can result in high energy costs. Many existing methods to solve this problem use heuristics which do not consider containers leaving the data center after being allocated. Some do consider migrating containers between VMs but few do for energy efficiency reasons. These overlooked aspects may lead to increased energy usage, particularly since studies have demonstrated that many containers run for only a brief duration. In this paper, we develop a model of the container-based cloud resource allocation problem that considers the energy impact of leaving and migrating containers. We then design a new Genetic Programming Hyper-Heuristic (GPHH) algorithm to jointly evolve three heuristics for container placement, VM placement and container migration control. We utilize newly designed terminals to ensure the effectiveness of our GPHH algorithm. Experiments have been conducted with results indicating that the heuristics evolved by our GPHH algorithm can achieve better performance compared to several state-of-the-art techniques.

Internet of Things Service Composition (SCIoT) aims to find the best composite IoT service to fulfil users' requirements. Given the NP-hard complexity of SCIoT, Evolutionary Computation methods, especially Estimation of Distribution Algorithms (EDAs), have received increasing attention to solve SCIoT problems. As one of the most popular EDA methods, Population-Based Incremental Learning (PBIL) has demonstrated its strong competency in optimising composed services in SCIoT. However, conventional PBIL does not explicitly utilize problem knowledge such as QoS and service replication, limiting its effectiveness for IoT service composition. In this paper, we propose a new PBIL based approach, named Population-Based Incremental Learning to Improve Service Composition (PBILISC), to solve the SCIoT problem. Different from traditional PBIL, PBILISC seamlessly integrates PBIL with QoS-aware local search (QLS) to effectively handle replicated services in the SCIoT problem. Specifically, PBILISC evolves a series of populations of solutions jointly through PBIL and QLS. PBIL leverages a probability distribution for population updates, while QLS focuses on improving the best evolved solution by searching promising neighboring solutions under the guidance of QoS. Experimental results show that PBILISC can outperform PBIL and several state-of-the-art methods on multiple benchmark SCIoT problems.

Software-defined networking (SDN) allows flexible and centralized control in cloud data centers. An elastic set of distributed SDN controllers is often required to provide sufficient yet cost-effective processing capacity. However, this introduces a new challenge: Request Dispatching among the controllers by SDN switches. It is essential to design a dispatching policy for each switch to guide the request distribution. Existing policies are designed under certain assumptions, including a single centralized agent, global network knowledge, and a fixed number of controllers, which often cannot be satisfied in practice. This article proposes MADRina, Multiagent Deep Reinforcement Learning for request dispatching, to design policies with high dispatching adaptability and performance. First, we design a multiagent system to address the limitation of using a centralized agent with global network knowledge. Second, we propose a Deep Neural Network-based adaptive policy to enable request dispatching over an elastic set of controllers. Third, we develop a new algorithm to train the adaptive policies in a multiagent context. We prototype MADRina and build a simulation tool to evaluate its performance using real-world network data and topology. The results show that MADRina can significantly reduce response time by up to 30% compared to existing approaches.

Location-Aware Dynamic Online Application dePloyment (LADOAP) in clouds is an NP-hard combinatorial optimisation problem. Genetic Programming Hyper-Heuristic (GPHH) has emerged as a promising approach for addressing LADOAP demands by dynamically generating Virtual Machine (VM) selection heuristics online. However, the performance of GPHH is impeded by long simulation times and low sampling efficiency. In this paper, we propose a novel hyper-heuristic framework that integrates Genetic Programming Hyper-Heuristic (GPHH) and Reinforcement Learning (RL) approaches to evolve rules for efficiently selecting location-aware Virtual Machines (VMs) capable of hosting multiple containers. The RL policy's value function acts as a surrogate model, significantly expediting the evaluation of generated VM selection rules. By applying this hybrid framework to LADOAP problems, we achieve competitive performance with a notable reduction in the number of required simulations. This innovative approach not only enhances the efficiency of VM selection but also contributes to advancing the state-of-the-art in addressing complex LADOAP challenges.

Double row layout problem (DRLP) involves identifying the exact locations of machines participating in a production task on two rows. There are typically multiple layouts with approximately optimal material handling cost for a DRLP. These layouts often exhibit significantly different layout configurations. Identifying multiple global or local optimal layouts can provide layout designers with a wide range of options, which is of great significance for enhancing the maintainability, scalability, and customisability of the facility. However, most existing studies on DRLPs typically focus on designing a single optimal layout. In this paper, we study a multi-modal optimization of double row layout problem (MDRLP). A hybrid approach combing a fast niching memetic algorithm and linear programming (FNMA-LP) is proposed for MDRLP to locate multiple global or local optimal layouts with a similar quality. First, a fast niching memetic algorithm is developed to find a set of approximate optimal machine sequences. Then, LP is employed to optimise the exact locations of machines for each machine sequence. To evaluate the performance of the proposed algorithm, FNMA-LP is compared against three popular multi-modal algorithms and a state-of-the-art single-modal algorithm developed for DRLP. Experiments show that our approach outperforms competing approaches on almost all problem instances.

Internet of Things (IoT) devices have become a crucial part of daily life. Because IoT devices often have small processing capability and low power supply, two popular technologies, i.e. cloud servers and fog edges, are increasingly integrated with IoT for workflow execution, giving rise to the resource allocation and workflow scheduling problem in hybrid IoT environments, i.e. the IoT workflow scheduling (IoTWS) problem. To tackle this NP-hard IoTWS problem, a new Genetic Algorithm (GA) called IoTGA has been successfully developed in this paper. In comparison to state-of-the-art GA approaches from literature, IoTGA allows fast workflow execution and can explicitly reduce the time and energy consumption thanks to its use of a newly designed local search method. Experiments on benchmark IoTWS problems clearly indicate that IoTGA can significantly outperform several competing GA methods and are more useful in practice.

Containers are quickly gaining popularity in cloud computing environments due to their scalable and lightweight characteristics. However, the problem of Resource Allocation in Container-based clouds (RAC) is much more challenging than the Virtual Machines (VMs)based clouds because RAC includes two levels of allocation problems: allocating containers to VMs and allocating VMs to Physical Machine (PMs). In this paper, we proposed a novel Group Genetic Algorithm (GGA) with energy-aware crossover, Best-Fit-Decreasing Insert (BFDI), and Local Search based Unpack (LSU) operator to solve RAC problems. Meanwhile, we apply an energy model with heterogeneous PMs that accurately captures the energy consumption of cloud data centers. Compared to state-of-the-art methods, experiments show that our method can significantly reduce the energy consumption on a wide range of test datasets.