The Multi-Agile Earth Observation Satellite Scheduling Problem (MAEOSSP) aims to maximize the total observation profits of a multi-satellite system while satisfying temporal and resource constraints. This problem has gained prominence owing to the increasing complexity of task scheduling scenarios and challenges in satellite management and control. MAEOSSP typically consists of two decision-making stages: task allocation and single-satellite scheduling. However, its NP-hard characteristic renders traditional methods susceptible to problem instance variations and computationally intensive. To address these issues, we propose the TAM-ECH, which integrates a Task Allocation Model (TAM) and an Ensemble Construction Heuristic (ECH) to solve MAEOSSP. TAM utilizes a Deep Q-learning Network (DQN) to efficiently allocate tasks to each satellite, while ECH employs ensemble heuristics for fast and stable single-satellite scheduling. Experimental results demonstrate that TAM-ECH outperforms state-of-the-art algorithms in both optimization speed and quality. On average, it achieves a profit rate 6% higher than GRILS and 30% higher than A-ALNS, with reduced solving time. Further experiments validate TAM's efficiency in task allocation and ECH's ability to provide stable, high-quality foundational assistance. Therefore, TAM-ECH provides an efficient solution to MAEOSSP, demonstrating its potential for application in upcoming large constellations and new management modes.

The agile earth observation satellite scheduling problem (AEOSSP) is a complex problem focused on maximizing observation rewards by optimizing the task observation sequence and the specific start times for each task. Existing methods have faced challenges in striking a balance between computational efficiency and solution quality. In this paper, we present a hybrid repair construction method (HRCM) that integrates baseline policy models (BPMs) and incorporates a slack and insertion (SI) mechanism to effectively address this challenge. Our method begins with the utilization of the BPM as an initial solution construction method, in which we design an observation construction model (OCM). Subsequently, the SI mechanism is introduced to rectify decision-making errors that may occur during the initial construction process. Experimental results present the significant enhancements in solution quality achieved by HRCM. Furthermore, we conduct a rigorous validation of the contributions made by OCM and SI to the overall improvement in solution quality. This research presents a promising method for enhancing the efficiency of earth observation satellite scheduling, with potential applications in various fields requiring optimal resource allocation and task sequencing.

Unmanned aerial vehicles (UAVs) are advanced flight systems. However, their positioning systems cause distance-dependent errors during flight. This study seeks to solve the UAV path planning problem with positioning error correction (UPEC) with an end-to-end method. Traditional methods struggle to balance solution quality and computational overload, and often have limited utilisation of scenario information. To overcome these issues, we propose a path planning model (PPM) based on deep reinforcement learning to solve the UPEC. The model has a complete structure that includes a mathematical model, feature engineering, solution process, neural policy network, scenario generation, training process, and test solution mechanism. Specifically, we first establish a Markov decision process (MDP) for UPEC and apply feature engineering with effective features to support decision-making. We then introduce a path planning neural network (PPNN) to represent the MDP policy. Based on the dataset generated from the multi-rule combination validation, we train the PPNN using the proposed RL algorithm with storage pool. Furthermore, we propose a backtracking mechanism to guarantee solution feasibility during the construction process. Extensive experiments demonstrate that the proposed PPM outperforms existing state-of-the-art algorithms in terms of solution quality and timeliness, and the backtracking mechanism effectively improves the scenario completion rate. The model study indicates the efficacy of our training algorithm and the generalisation of the PPNN. Additionally, our construction process is problem-tailored and more suitable for addressing UPEC than iterative search algorithms, because it effectively mitigates the impact of invalid nodes.

The agile earth observation satellite scheduling problem (AEOSSP) with time-dependent transition times is a complex combinational optimization problem that has emerged from the development of large-scale satellite management techniques. To address this problem, we propose a deep reinforcement learning-based construction model (DRL-CM) that consists of five parts: 1) a Markov decision process (MDP); 2) a feature engineering; 3) a constructive heuristic neural network (CHNN); 4) an RL training method; and 5) an evaluation system. Specifically, the CHNN comprises six modules containing three special components that we propose: a dynamic encoder, a dynamic global layer, and a two-stage attention layer. First, we build the MDP of the AEOSSP and the feature engineering with effective features required for decision-making. Second, we design the CHNN to function as the MDP policy and train it with an RL model. Finally, we propose a comprehensive evaluation system for the validation of our model. The experimental results indicate that the proposed DRL-CM outperforms the state-of-the-art algorithm in terms of both optimization speed and quality. In addition, the feature engineering and network architecture built in our model are verified to be effective in comprehensive experiments.

The multi-objective dynamic agile earth observation satellite scheduling problem (MO-DAEOSSP) aims to schedule a set of real-time arrival requests and form a reasonable observation plan to satisfy various criteria. According to the requirements in practical applications, the total profit and the average image quality of scheduled requests are taken as optimization goals in this study. Compared to manually designed heuristics and iterative-based methods used in previous research, genetic programming based hyper heuristics (GPHH) can automatically evolve high-quality heuristic rules (HRs) for real-time scheduling without being highly dependent on expert knowledge. In this paper, a knowledge-transfer based multi-objective GPHH framework (KT-MOGP) is proposed, equipped with a heuristic-based simulation considering the idle monitoring, to evolve non-dominated HRs for solving MO-DAEOSSP. The heuristic-based simulation generates feasible schedules and returns fitness values for given HRs, which are the individuals evolved by KT-MOGP. KT-MOGP applies a knowledge transfer mechanism to accelerate convergence. Once a source problem is trained, its non-dominated solutions are extracted and their feature importance is transferred to guide the initialization of another target problem, by which the knowledge generated during the training process can be fully utilized. Experimental results on three sets of instances show that KT-MOGP outperforms the existing GPHH-based method and that the evolved HRs are competitive compared to several classical constructive heuristics and multi-objective evolutionary algorithms. The results also show the effectiveness of the proposed knowledge transfer-based initialization. To the best of our knowledge, this study is the first attempt to consider both multi-objective scenarios and real-time arrival requests.

To improve the computational efficiency of multi-fidelity simulation, an improved fully coupled method was proposed, which was applied to the coupled simulation of a 3-D model of an exhaust system and a 0-D model of a turbofan engine. Different from conventional approach, by “truncating” the engine model, the CFD calculation of the exhaust system can be decoupled from the solution of nonlinear equations in the engine model, and an outer iteration equation set was constructed to solve the multi-fidelity model. It was found that the improved fully coupled method has better computational efficiency than traditional methods while maintaining equivalent computational results. Then, the improved fully coupled method was applied to the design of the central cone film cooling system, the influence of cooling gas bleed mechanism, the hole open area ratio of the central cone and hole density, hole direction, hole shape on the infrared characteristics was investigated. The impact of central cone film cooling on the overall performance of the engine was quantitatively evaluated. Combined with various favorable factors, when maintaining the HP rotor speed unchanged, the integrated infrared radiation intensity of the central cone under supersonic cruise conditions is reduced by 93 % compared to the uncooled state, while the engine thrust increases by 0.3 % and fuel consumption rate increases by 0.18 %. The multi-fidelity simulation method proposed in this paper and the conclusions obtained are of great significance for the structural design of low-infrared exhaust systems.

To ensure that the aerothermodynamic cycle design of a turbofan engine is more accurate, efficient, and provide a reliable decision-making basis for engine designers, the multi-objective particle swarm optimization (MOPSO) method was used to optimize the aerothermodynamic performance parameters of the turbofan engine at multiple design points (MDPs). Fuel consumption rate and the specific thrust were considered as optimization targets. The thrust requirements and cycle parameter constraints under each working state were comprehensively considered to obtain the optimal performance boundary of the engine, the corresponding cycle parameters, and the correlations between different requirements and constraints. The results showed that the MOPSO algorithm could accurately and completely obtain the optimal performance boundary surface of the engine in the feasible region and the corresponding cycle parameter value. The feasible region obtained by the aerothermodynamic cycle design at MDPs was more accurate and effective than the design at a single design point.

The agile earth observation satellite scheduling problem (AEOSSP), as a time-dependent and arduous combinatorial optimization problem, has been intensively studied in the past decades. Many studies have proposed non-iterative heuristic construction algorithms and iterative meta-heuristic algorithms to solve this problem. However, the heuristic construction algorithms spend a relatively shorter time at the expense of solution quality, while the iterative meta-heuristic algorithms accomplish a high-quality solution with a lot of time. To overcome the shortcomings of these approaches and efficiently utilize the historical scheduling information and task characteristics, this paper introduces a new neural network model based on the deep reinforcement learning and heuristic algorithm (DRL-HA) to the AEOSSP and proposes an innovative non-iterative heuristic algorithm. The DRL-HA is composed of a heuristic construction neural network (HCNN) model and a task arrangement algorithm (TAA), where the HCNN aims to generate the task planning sequence and the TAA generates the final feasible scheduling order of tasks. In this study, the DRL-HA is examined with other heuristic algorithms by a series of experiments. The results demonstrate that the DRL-HA outperforms competitors and HCNN possesses outstanding generalization ability for different scenario sizes and task distributions. Furthermore, HCNN, when used for generating initial solutions of meta-heuristic algorithms, can achieve improved profits and accelerate interactions. Therefore, the DRL-HA algorithm is verified to be an effective method for solving AEOSSP. In this way, the high-profit and high-timeliness of agile satellite scheduling can be guaranteed, and the solution of AEOSSP is further explored and improved.

The agile earth observation satellite scheduling problem (AEOSSP) consists of selecting and scheduling a number of tasks from a set of user requests in order to optimize one or multiple criteria. In this paper, we consider a multi-objective version of AEOSSP (called MO-AEOSSP) where the failure rate and the timeliness of scheduled requests are optimized simultaneously. Due to its NP-hardness, traditional iterative problem-tailored heuristic methods are sensitive to problem instances and require massive computational overhead. We thus propose a deep reinforcement learning and parameter transfer based approach (RLPT) to tackle the MO-AEOSSP in a non-iterative manner. RLPT first decomposes the MO-AEOSSP into a number of scalarized sub-problems by a weight sum approach where each sub-problem can be formulated as a Markov Decision Process (MDP). RLPT then applies an encoder–decoder structure neural network (NN) trained by a deep reinforcement learning procedure to producing a high-quality schedule for each sub-problem. The resulting schedules of all scalarized sub-problems form an approximate pareto front for the MO-AEOSSP. Once a NN of a subproblem is trained, RLPT applies a parameter transfer strategy to reducing the training expenses for its neighboring sub-problems. Experimental results on a large set of randomly generated instances show that RLPT outperforms three classical multi-objective evolutionary algorithms (MOEAs) in terms of solution quality, solution distribution and computational efficiency. Results on various-size instances also show that RLPT is highly general and scalable. To the best of our knowledge, this study is the first attempt that applies deep reinforcement learning to a satellite scheduling problem considering multiple objectives.

The unconstrained binary quadratic programming (UBQP) problem is a difficult combinatorial optimization problem that has been intensively studied in the past decades. Due to its NP-hardness, many heuristic algorithms have been developed for the solution of the UBQP. These algorithms are usually problem-tailored, which lack generality and scalability. To address these issues, a heuristic algorithm based on deep reinforcement learning (DRLH) is proposed in this paper. It features in inputting specific features and using a neural network model called NN to guild the selection of variable at each solution construction step. Also, to improve the algorithm speed and efficiency, two algorithm variants named simplified DRLH (DRLS) and DRLS with hill climbing (DRLS-HC) are developed as well. These three algorithms are examined through extensive experiments in comparison with famous heuristic algorithms from the literature. Experimental results show that the DRLH, DRLS, and DRLS-HC outperform their competitors in terms of both solution quality and computational efficiency. Precisely, the DRLH achieves the best-quality results, while DRLS offers a high-quality solution in a very short time. By adding a hill-climbing procedure to DRLS, the resulting DRLS-HC algorithm is able to obtain almost the same quality result as DRLH with however 5 times less computing time on average. We conducted additional experiments on large-scale instances and various data distributions to verify the generality and scalability of the proposed algorithms, and the results on benchmark instances indicate the ability of the algorithms to be applied to practical problems.

Satellite establishes a satellite ground station to satellite link with a ground station to complete data transmission. However, in China, satellite ground stations only exist within the country. With the rapid increase in the number of satellites, effectively dispatching satellite communications and maximizing the performance of ground stations are necessary. We propose a heuristic adaptive large neighborhood search algorithm (H-ALNS) to solve the satellite data transmission scheduling (SDTS) problem. The algorithm includes two heuristic rules for generating the initial scheme and the conflict reduction process after updating the neighborhood. A heuristic task assignment method is used to select the execution time window and position for the task sequence. The adaptive operator is used to update the delete operator and insert operator weights into the H-ALNS. The quality of the generated planning scheme improved through continuous neighborhood destruction and repair. Through experimental analysis, the algorithm we propose is feasible for solving SDTS problems and surpasses the planning results acquired using other comparison algorithms. The H-ALNS has good prospects for practical engineering applications.

Multi-Satellite Tracking Telemetry and Command Scheduling Problem is a multi-constrained, high-conflict complex combinatorial optimization problem. How to effectively utilize existing resources has always been an important topic in the satellite field. To solve this problem, this paper abstracts and simplifies the Multi-Satellite TT&C Scheduling problem and establishes the corresponding mathematical model. The hybrid goal of maximizing the profit and task completion rate is our objective function. Since the genetic algorithm has a significant effect in solving the problem of resource allocation, we have proposed an improved genetic algorithm with population perturbation and elimination (GA-PS) based on the characteristics of the Multi-Satellite TT&C Scheduling problem. A series of simulation experiments were carried out with the total profits and the task completion rate as the index of the algorithm. The experiment shows that compared with the other three comparison algorithms, our algorithm has better performance in both profit and task completion rate.

Agile Earth Observation Satellite Scheduling Problem (AEOSSP) consists in selecting a subset of tasks from a given task set, which is then scheduled on agile satellite, to maximize the total reward of scheduled tasks. AEOSSP is a NP- hard problem and the existing solving methods mainly focus on the field of heuristic and meta-heuristic method, Theoretically, it is impossible to find a single heuristic method that works well on any problem instance. In this paper, inspired by RNN and the attention mechanism, we abstracted the problem from fixed scenarios and proposed an end-to-end framework based on deep reinforcement learning. This model treats neural network as a complex heuristic method constructed by observing reward signals and following feasible rules. The trained model can directly obtain a scheduling sequence without retraining each new problem instance. Compared with the general heuristic rules, experiments prove that this method is more effective and more robust.

Flexible job shop scheduling problem with combined processing constraint is a common scheduling problem in assembly manufacturing industry. However, traditional methods for classic flexible job shop scheduling problem (FJSP) cannot be directly applied. To address this problem, the concepts of `combined processing constraint' and 'virtual operation' are studied and introduced to simplify and transform FJSP with combined processing constraint into FJSP. A Multi-agent system (MAS) for FJSP is used for fitting the requirement of building complex, flexible, robust and dynamic manufacturing scheduling. On this basis, a novel adaptive real-time scheduling method for MAS is further proposed for better adaptability and performance. This method solves the previously converted problem and conquers the shortcoming of poor performance of traditional single dispatching rule method in MAS. In this approach, the scheduling process is modeled as contextual bandit, so that each job agent can select the most suitable dispatching rules according to the environment state after learning to achieve scheduling optimization. The proposed method is compared with some common dispatching rules that have been widely used in MAS. Results illustrate the high performance of the proposed method in a simulated environment.

With the improvement of consumption concept, customer’s demand is becoming more and more dynamic, diversified and personalized, which requires the manufacturing system to be more and more robust and real-time. To achieve such purpose, in this paper, a hierarchical heterogeneous hybrid multi-agent system (MAS) based control architecture is proposed, a MAS-based shop floor scheduling model is designed, an event-driven rolling horizon scheduling mechanism is selected as the scheduling mechanism of the system and an improved contract network protocol is proposed to act as the system’s scheduling algorithm. According to the simulation experiments based on the ever built experimental platform, Internet of Things (IoT) based manufacturing real-time scheduling system is proven to be feasible. DOI: http://dx.doi.org/10.5755/j01.mech.24.1.18631