Parallel training of large-scale networks has attracted the attention of both artificial intelligence and high-performance distributed systems. One of efficient parallelism is the micro-batch-based pipeline, e.g., GPipe. Based on the GPipe, we derive a time-cost model with the basic time function of layers, which considers computing time and communication time simultaneously as well as treats these time as nonlinear to batch size. Focusing on the optimal solutions of network division and data partition, we propose a Cross-Search algorithm with Improved Multi-dimensional Dichotomy (CSIMD). Through theoretical derivation, we prove IMD has appreciable theoretical optimality. Also extensive experiments on both CNN- and Transformer-based networks demonstrate our proposed CSIMD can obtain optimal network division and data partition schemes under GPipe parallelism: CSIMD achieves training speeds respectively 2.0× and 2.5× faster than GPipe-R and GPipe-E in CNNs; as well as 1.5× and 1.6× in Transformers.

As the quantity and complexity of information processed by software systems increase, large-scale software systems have an increasing requirement for high-performance distributed computing systems. With the acceleration of the Internet in Web 2.0, Cloud computing as a paradigm to provide dynamic, uncertain and elastic services has shown superiorities to meet the computing needs dynamically. Without an appropriate scheduling approach, extensive Cloud computing may cause high energy consumptions and high cost, in addition that high energy consumption will cause massive carbon dioxide emissions. Moreover, inappropriate scheduling will reduce the service life of physical devices as well as increase response time to users' request. Hence, efficient scheduling of resource or optimal allocation of request, that usually a NP-hard problem, is one of the prominent issues in emerging trends of Cloud computing. Focusing on improving quality of service (QoS), reducing cost and abating contamination, researchers have conducted extensive work on resource scheduling problems of Cloud computing over years. Nevertheless, growing complexity of Cloud computing, that the super-massive distributed system, is limiting the application of scheduling approaches. Machine learning, a utility method to tackle problems in complex scenes, is used to resolve the resource scheduling of Cloud computing as an innovative idea in recent years. Deep reinforcement learning (DRL), a combination of deep learning (DL) and reinforcement learning (RL), is one branch of the machine learning and has a considerable prospect in resource scheduling of Cloud computing. This paper surveys the methods of resource scheduling with focus on DRL-based scheduling approaches in Cloud computing, also reviews the application of DRL as well as discusses challenges and future directions of DRL in scheduling of Cloud computing.

Optimizing multi-dimensional resource utilization is a critical research area in distributed computing, particularly in cloud computing, where various heterogeneous resources are integrated to offer a wide range of services. Addressing this issue necessitates the simultaneous consideration of multiple resource bottlenecks. This paper presents a new solution, called the Multi-Population Growth Genetic Algorithm (MPGGA), which consists of a central management unit responsible for executing information interaction and growth quota reallocation, and multiple population evolution executors to perform crossover and regeneration within each population. The proposed MPGGA combines elite sharing and priority support for the weaker population (ESPW), resulting in better convergence and optimality than other combinations of strategies. This outcome is corroborated by extensive ablation experiments on various strategies. Furthermore, the experimental results for minimizing the maximum utilization of resources in each dimension indicate that MPGGA-ESPW outperforms other popular algorithms, such as GHW-NSGA II (1.363x), GHW-MOEA/D (1.339x), NSGA II (1.948x), and MOEA/D (2.151x) in terms of convergence speed. For energy consumption-related optimization problems, the experimental results demonstrate that the adaptability of a single algorithm in MPGGA family is limited by the algorithm of growth route, while also showing that the MPGGA framework is flexible to allow various algorithms as its growth route to adapt to various scenarios.

Facial expression recognition (FER) in the wild is a challenging pattern recognition task affected by the images' low quality and has attracted broad interest in computer vision. Existing FER methods failed to obtain sufficient accuracy to support the practical applications, especially in scenarios with low fault tolerance, which limits the adaptability of FER. Targeting exploring the possibility of further improving the accuracy of FER in the wild, this paper proposes a novel single model named R18+FAML and an ensemble model named R18+FAML-FGA-T2V, which applies intra-feature fusion within a single network, feature fusion among multiple networks, and the ensemble decision strategy. Based on the backbone of ResNet18 (R18), R18+FAML combines internal feature fusion and three attention blocks, as well as uses multiple loss functions (FAML) to improve the diversity of the feature extraction. To effectively integrate feature extractors from multiple networks, we propose feature fusion among networks based on the genetic algorithm (FGA). Comprehensively considering and utilizing more classification information, we propose an ensemble strategy, i.e., the improved top-two-voting (T2V) of multiple networks with the same structure. Combining the above strategies, R18+FAML-FGA-T2V can focus on the main expression-aware areas by integrating interest areas of multiple networks. From experiments on three challenging FER datasets in the wild including RAF-DB, AffectNet-8 and AffectNet-7, our single model R18+FAML and ensemble model R18+FAML-FGA-T2V achieve the accuracies of 90.32,62.17,65.83% and 91.59,63.27,66.63% respectively, both achieving the state-of-the-art results.

Cloud computing, as a large-scale distributed computing system dynamically providing elastic services, is designed to meet the requirement of delivering computing services to users as subscription-oriented services. In general, the problems of resource scheduling in Cloud computing like minimizing makespan are usually NP-Hard problems. Various common algorithms including heuristic, meta-heuristic and machine learning are applied in resource scheduling of Cloud computing to obtain the solutions, which however are still probable and imperative to be optimized. Through innovatively applying heuristic algorithms namely LPT (Longest Processing Time) and BFD (Best Fit Decreasing) as the basic search routes and integrating these with neighborhood search algorithm namely OneStep, this paper proposes multi-search-routes-based algorithms containing LPT-OneStep, BFD-OneStep and their combinations for the sake of enhancing theoretical performance and improving solutions of scheduling schemes especially for problems of minimizing makespan for homogeneous and heterogeneous resources. Theoretical derivations prove that the proposed algorithms possess better theoretical approximation ratios for P||Cmax. Extensive experiments on simulation environment demonstrate the proposed algorithms outperform than corresponding compared algorithms for minimizing makespan problems in both homogenous resources and heterogeneous resources, which validates the superiority of the proposed algorithms.

Load balancing is vital for the efficient and long-term operation of cloud data centers. With virtualization, post (reactive) migration of virtual machines (VMs) after allocation is the traditional way for load balancing and consolidation. However, it is not easy for reactive migration to obtain predefined load balance objectives and it may interrupt services and bring instability. Therefore, we provide a new approach, called Prepartition, for load balancing. It partitions a VM request into a few sub-requests sequentially with start time, end time and capacity demands, and treats each sub-request as a regular VM request. In this way, it can proactively set a bound for each VM request on each physical machine and makes the scheduler get ready before VM migration to obtain the predefined load balancing goal, which supports the resource allocation in a fine-grained manner. Simulations with real-world trace and synthetic data show that our proposed approach with offline version (PrepartitionOff) scheduling has 10%–20% better performance than the existing load balancing baselines under several metrics, including average utilization, imbalance degree, makespan and Capacity_makespan. We also extend Prepartition to online load balancing. Evaluation results show that our proposed approach also outperforms state-of-the-art online algorithms.

Multi-Dimensional Resources Scheduling Problem (MDRSP, usually a multi-objective optimization problem) has attracted focus in the management of large-scale cloud computing systems as the collaborative operation of various devices in the cloud affects resource utilization and energy consumption. Effective management of the cloud requires a higher performance method to solve MDRSP. Considering the complex coupling between multi-dimensional resources and focusing on virtual machines allocation, we propose GGA-HLSA-RW (GHW, a novel family of genetic algorithms) to optimize the utilization and energy consumption of the cloud. In GGA-HLSA-RW, we add a growth stage to the genetic algorithm and construct a Growable Genetic Algorithm (GGA) using the Heuristic-based Local Search Algorithm (HLSA) with Random multi-Weights (RW) as the growth route. Based on the GHW, we propose GHW-NSGA II and GHW-MOEA/D by applying the sorting strategies and population regeneration mechanism of NSGA II and MOEA/D. To evaluate the performance of GHW, we carry out extensive experiments on the simulation dataset and AzureTraceforPacking2020 for the problems of minimizing the maximum utilization rate of resources for each dimension and minimizing total energy consumption. Experiment results demonstrate the advantages of growth strategy and dimensionality reduction strategy of GHW, as well as validate the applicability and optimality of GHW in realistic cloud computing. The experiments also demonstrate our proposed GHW-NSGA II and GHW-MOEA/D have better convergence rates and optimality than state-of-the-art NSGA II and MOEA/D.

Cloud computing contains large-scale tasks and resources. Currently, the local search is a considerable choice in ensuring both computational complexity and optimization. Based on our previous research on multi-route search algorithm to reduce makespan, we apply BFDO and LPTO algorithms to address load balancing and bin-packing problems. Through abundant experiments, we validate the superiority of our proposed BFDO and LPTO.

This study looked into the problems that occur when a clustering or cluster ensemble model with unsupervised or semi-supervised learning is used in a real-world setting. However, validating the obtained results is a challenging task. Therefore, we propose a self-directed learning (SDL) framework for cluster ensemble, which improves the traditional ensemble framework by assisting the consensus function in achieving the highest assessment of clustering performance. The SDL is built on two models: predicting test set labels (PTL) and detecting best results (DBR). The PTL model assists in predicting the test set labels; the PTL is based on consistently ensembling the outcomes until it obtains satisfactory results. The DBR identifies the correct answers for each data object when a single model provides several outcomes for a single dataset. To highlight the power of our proposed framework, we consider multiple performance measurements; one of them is termed as correction ratio (CR). We compare SDL with several cluster ensemble models. The results show that the proposed framework outperforms other models with 18% accuracy on average and achieves similar results for other performance indicators. (c) 2022 The Author(s). Published by Elsevier B.V. on behalf of King Saud University. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

Cloud computing environment is becoming increasingly complex due to its large-scale information growth and increasing heterogeneity of computing resources. Hierarchical Cloud computing dividing the system into multi-levels with multiple subsystems to support the adaptability to abundant requests from users has been widely applied and brings great challenges to resource scheduling. It is critical to find an effective way to address the complex scheduling problems in hierarchical Cloud computing, whose scenarios and optimization objectives often change with the types of subsystems. In this paper, we propose a scheduling framework to select the scheduling algorithms (SFSSA) for different scheduling scenarios considering no algorithm well suitable to all scenarios. To concretize SFSSA, we propose deep learning-based algorithms selectors (DLS) trained by labeled data and deep reinforcement learning-based algorithms selectors (DRLS) trained by feedback from dynamic scenarios to complete the algorithms selection regarding the scheduling algorithms as selectable tools. Then, we apply strategies including pre-trained model, long experience reply and joint training to improve the performance of DRLS. To enable the quantitative comparison of selectors, we introduce a weighted cost model for the trade-off between solution and complexity. Through multiple sets of experiments in hierarchical Cloud computing with multi subsystems for five types of scheduling problems and varying weights of cost, we demonstrate DLS and DRLS outperform baseline strategies. Compared with random selector, greedy selector, round-robin selector, single best selector, virtual best selector and single fast selector, DLS reduces the cost by 47.4%, 46.1%, 33.9%, 47.9%, 19.3%, 18.8% under stable parameter ranges, and DRLS reduces the cost by 41.1%, 40.6%, 11.7%, 42.3%, 11.5%, 12.5% in dynamic scenarios respectively. In experiments, we also validate DRLS has stronger adaptability than DLS in dynamic scheduling scenarios and DRLS using all of strategies achieves the best performance.

Cloud computing has rapidly emerged as a model for delivering Internet-based utility computing services. Infrastructure as a Service (IaaS) is one of the most important and rapidly growing models in cloud computing. Scalability, quality of service, optimum utility, decreased overheads, higher throughput, reduced latency, specialised environment, cost-effectiveness, and a streamlined interface are some of the essential elements of cloud computing for IaaS. Traditionally, resource management has been done through static policies, which impose certain limitations in various dynamic scenarios, prompting cloud service providers to adopt data-driven, machine-learning-based approaches. Machine learning is being used to handle various resource management tasks, including workload estimation, task scheduling, VM consolidation, resource optimisation, and energy optimisation, among others. This paper provides a detailed review of machine learning-based resource management solutions. We begin by introducing background concepts of cloud computing like service models, deployment models, and machine learning use in cloud computing. Then we look at resource management challenges in cloud computing, categorise them based on various aspects of resource management types such as workload prediction, VM consolidation, resource provisioning, VM placement and thermal management, review current techniques for addressing these challenges, and evaluate their key benefits and drawbacks. Finally, we propose prospective future research directions based on observed resource management challenges and shortcomings in current approaches for solving these challenges.

Many clustering and cluster ensemble models have been proposed recently and have not addressed two concerns; when a single model is executed multiple times on a dataset, it predicts various labels for each data object; however, these various labels have a small correctness ratio due to the randomness in generating values in each implementation. Further, detecting which the correct label from these diverse answers is complicated, specifically when the unsupervised model works on a real-world application and needs to deliver a single correct label to the user. In this work considered these two issues by proposing a novel unsupervised constraints termed Inherited Constraints (IC) that behaves as semi-supervised constraints generation. Moreover, execute the IC needs a cluster model to utilize; thus, we proposed an unsupervised cluster ensemble model by integrating the Density Peaks cluster ensemble framework (DPE) and IC to improve the performance. This model is termed DPEIC. Further, we proposed a model termed Answer Settlements (AS) to detect a single correct label for each data object from the diverse answers obtained by DPEIC after utilized multiple times to consider the most duplicated labels as the correct ones. We compare DPEIC-AS with several state-of-the-arts to validate the strengths of this work. The experimental results indicate that DPEIC-AS outperforms the compared models at a different rate, ranging from 3% to 93%. Also, The AS assisted two state-of-the-arts methods to detect the correct labels with the highest possibilities from diverse answers.

创新地运用瞬时等效转换矩阵描述船舶多自由度耦合的参数横摇运动,并考虑坐标轴瞬变建立多自由度耦合运动方程.以某C11集装箱船为例计算其在各波高下的随机纵浪中的运动响应,并通过非线性力学的分析方法对其运动特性进行分析.亦对实船下的不同波高进行多组计算,通过概率统计其幅值.引入风险评估的思路对其分布进行拟合和推导概率估算方法.并在此基础上继续对波高进行拟合得到其三维的概率分布函数,可估算随机纵浪中任意波高横摇角发生的概率.

To research the forced-parametrically excited rolling of ships , a new coordinate transformation method on an instantaneous equivalent rotation matrix is established to describe the multi -degree-of-freedom ( MDOF) motion of a ship in space.Based on the instantaneous equivalent motion system , an MDOF coupled motion equation that considers the transient position of the onboard coordinate system is established.The heading control condition is in-troduced and the MDOF motion equation is improved to study the forced-parametrically excited rolling of ships un-der directional navigation in regular oblique waves.The movement of a C11 container ship in a regular oblique wave and different wave conditions is analyzed , which reveals that there is a relatively fixed time delay of phase in the motion with a wavelet angle difference.The influences of the parametric excitation and strong excitation on the mo-tion are related to the wave direction angle , wave amplitude, and wavelength, and these wave conditions also deter-mine if the natural frequency influences the motion frequency when the forced-parametric excitation occurs.

Parametric rolling is the central issue of complete stability.In this paper,instantaneous equivalent rotation matrix is used to describe the rotation of the ship around three axes.Formula for numerical calculation of grid feature is deduced,which can be used to the underwater part.Then,formula for numerical calculation of real-time wave force can be obtained.A procedure of motion in the time domain is compiled,which can calculate motion of MDOF in waves.Taking a C11 container ship as an example,its motion in a variety of waves is calculated,and motion characteristics are analyzed according to the phase diagram or spectrum.Feasibility and convergence of the numerical calculation method in the time domain are verified,and multiple coupling responses are analyzed.