A combustion chamber pressure- and mixture ratio-controller based on deep reinforcement learning methods is under development at German Aerospace Center - DLR. Training and implementation of the controller is described. The performance of this controller is evaluated in simulation and experimental tests. Application use case is a 22N nitrous oxide/ethane thruster that can be operated in a wide range of operation points. Steady state and trajectory-following performance is measured via mean squared error, standard deviation, overshot and settling time evaluation in simulation and experiment. For steady state set-point control the controller shows good performance within the trained operation points in simulation and experiment. Root mean squared error of chamber pressure and mixture ratio is below 0.5 bar and 1 respectively for all physically achievable set-points. Control performance in simulation is better than at the real system. Trajectory-following performance in simulation is convincing, showing nearly no offset to the target values, whereas in the experimental evaluation deviations and fluctuations occur, indicating the need for further optimization when transferring the controller from simulation to the real system.

At DLR neural networks, as potential future controller for rocket engines, are studied. A neural network-based chamber pressure controller for a simplified cold gas thruster is presented and analyzed in simulation and experiment. The goal of the controller is twofold: It can track a trajectory with different changes of setpoints and it allows to set and control a wide variety of steady state chamber pressures. The neural network gets feeding line pressure measurement data as input and calculates valve positions as output values. The training phase of the controller is done with a reinforcement learning algorithm in an EcosimPro/ESPSS simulation, that is validated with data from the corresponding experimental set up. To increase the robustness and to allow a transfer from the simulation directly to the test facility domain randomization is applied. The controller is evaluated in simulations and experiment. It was found that - in the range of physically possible operation points - the controller achieves a constantly high reward which corresponds to a low error and a good control performance. In the simulation the controller was able to adjust all required set points with a steady state error of less than 0.1bar while retaining a small overshoot and an optimal settling time. It is found that the controller is also able to regulate all desired set points in the real experiment. A reference trajectory with different steps, linear and sinus changes in target pressure is tested in simulation and experiment. The controller was in both cases able to successfully follow the given trajectory.

The DLR Liquid Upper Stage Demonstrator Engine (LUMEN) is an experimental expanderbleed cycle rocket engine designed for LOX/LNG propulsion in the 25 kN thrust range. Its primary purpose is to serve as a flexible testing platform for industrial and institutional partners, focusing on addressing the challenges associated with rocket engine reusability like health monitoring and intelligent control algorithms. The LUMEN demonstrator allows for sub-component exchange and testing. This approach simplifies for example the turbopump design by incorporating two turbopumps instead of a single shaft configuration, providing greater flexibility for technology development. In addition to component technology maturation, the modular system approach enables the testing of technologies that would otherwise be challenging to assess in full-scale or simulation environments, such as health management and its feedback to engine control. The closed-loop neural network-based control improves the accuracy of approximating the nonlinear behavior of the engine. Finally, the upcoming development steps for LUMEN are outlined.

Traditional experimental design methods often face challenges in handling complex aerospace systems due to the high dimensionality and nonlinear behavior of such systems, resulting in nonoptimal experimental designs. To address these challenges, machine learning techniques can be used to further increase the application areas of modern Bayesian Optimal Experimental Design (BOED) approaches, enhancing their efficiency and accuracy. The proposed method leverages neural networks as surrogate models to approximate the underlying physical processes, thereby reducing computational costs and allowing for full differentiability. Additionally, the use of reinforcement learning enables the optimization of sequential designs and essential real-time capability. Our framework is validated by optimizing experimental designs that are used for the efficient characterization of turbopumps for liquid propellant rocket engines. The reinforcement learning approach yields superior results in terms of the expected information gain related to a sequence of 15 experiments, exhibiting mean performance increases of 9.07% compared to random designs and 6.47% compared to state-of-the-art approaches. Therefore, the results demonstrate significant improvements in experimental efficiency and accuracy compared to conventional methods. This work provides a robust framework for the application of advanced BOED methods in aerospace testing, with implications for broader engineering applications.

Advanced control algorithms will change the way we operate liquid propellant rocket engines in the future. One approach is to use neural networks as the control policy, which are trained in simulation with deep reinforcement learning. Neural network controllers can easily deal with multiple high-level control goals such as maximum efficiency, long fatigue-life, or mechanical and thermodynamic constraints. Within its technological road map towards cost-efficient and reusable engines, the German Aerospace Center (DLR) is developing an advanced engine controller for an expander-bleed engine. In this work, we design a neural network controller for the LUMEN turbopump and test the control performance at our test bench, tackling the challenges of real-world systems.

Accurate calculations of the heat transfer and the resulting maximum wall temperature are essential for the optimal design of reliable and efficient regenerative cooling systems. However, predicting the heat transfer of supercritical methane flowing in cooling channels of a regeneratively cooled rocket combustor presents a significant challenge. High-fidelity CFD calculations provide sufficient accuracy but are computationally too expensive to be used within elaborate design optimization routines. In a previous work it has been shown that a surrogate model based on neural networks is able to predict the maximum wall temperature along straight cooling channels with convincing precision when trained with data from CFD simulations for simple cooling channel segments. In this paper, the methodology is extended to cooling channels with curvature. The predictions of the extended model are tested against CFD simulations with different boundary conditions for the representative LUMEN combustor contour with varying geometries and heat flux densities. The high accuracy of the extended model's predictions, suggests that it will be a valuable tool for designing and analyzing regenerative cooling systems with greater efficiency and effectiveness.

The German Aerospace Center (DLR) Institute of Space Propulsion has unique expertise in operating test facilities for rocket engine testing and development in Europe since 1959. However, essential elements of the test site were designed up to half a century ago. In order to ensure a futureproof and intelligent digital test infrastructure, the potential of test automation, advanced control, and monitoring systems is investigated based on machine learning. Such intelligent control systems are expected to reduce engine development and test preparation times, thereby lowering the associated costs. Additionally, advanced monitoring systems are anticipated to increase the safety and reliability of the test infrastructure. This paper presents the results of two pilot projects: the first project uses reinforcement learning to automatically generate test sequences based on test requirements, while the second project develops a feed-forward forecasting model to predict deviations from expected behavior in the feed-line of a rocket engine test facility.

The last years have witnessed an enormous interest in the use of artificial intelligence methods, especially machine learning algorithms. This also has a major impact on aerospace engineering in general, and the design and operation of liquid rocket engines in particular, and research in this area is growing rapidly. The paper describes current machine learning applications at the DLR Institute of Space Propulsion. Not only applications in the field of modeling are presented, but also convincing results that prove the capabilities of machine learning methods for control and condition monitoring are described in detail. Furthermore, the advantages and disadvantages of the presented methods as well as current and future research directions are discussed.

The regenerative cooling still is one of the most crucial parts in designing a liquid rocket engine. The goal of an effective design is a trade-off between a sufficient cooling of the structure and a low pressure drop in the cooling channels. For an expander type cycle a third requirement arises: A sufficient enthalpy increase of the cooling fluid for the turbopumps to operate effectively. To prevent the combustion chamber from melting, the regenerative cooling has to be designed with a sufficient margin to respect all uncertainties. A neural network based surrogate model is used to study the robustness of the cooling channel design for other load points of the engine. In this paper, the design of the regenerative cooling system and the manufacturing of a subscale combustion chamber will be discussed.

Intelligent engine control could be one of the most important innovations in the development of future reusable engines, facilitating a safer and more economical engine operation. In this work, we investigate the closed-loop control of the LUMEN expander-bleed engine by combining machine learning with a transient simulation environment. The controller can dynamically change the set-point of the engine between a chamber pressure of 40 bar to 80 bar by adjusting up to six flow control valves while maintaining several boundary conditions at any given time.

Advanced engine control is an important requirement for the efficient operation of future reusable engines, facilitating a safer and more economical engine operation. For this reason, modern control strategies are extensively studied in recent years, mainly through the use of simulation environments. An important development step is to test the performance and robustness of the control algorithm at real test benches. The present paper describes the first steps towards the use of an reinforcement learning based controller on a N2O C2H6 22N green propellant thruster. The control objectives are given by regulating the mixture ratio and combustion pressure. The existing test bench is modelled in EcosimPro\,/\,ESPSS. Based on the simulation model deep reinforcement learning is used to train the controller and domain randomization is used to increase the robustness. The overall goal is to transfer the controller from the simulation model to the real test bench. Finally, preliminary experiments demonstrate the basic functionality of reinforcement learning based controllers for real rocket propulsion systems.

Nowadays, liquid rocket engines use closed-loop control at most near-steady operating conditions. The control of the transient phases is traditionally performed in open loop due to highly nonlinear system dynamics. This situation is unsatisfactory, in particular for reusable engines. The open-loop control system cannot provide optimal engine performance due to external disturbances or the degeneration of engine components over time. In this article, we study a deep reinforcement learning approach for optimal control of a generic gas-generator engine's continuous startup phase. It is shown that the learned policy can reach different steady-state operating points and convincingly adapt to changing system parameters. Compared to carefully tuned open-loop sequences and proportional-integral-derivative (PID) controllers, the deep reinforcement learning controller achieves the highest performance. In addition, it requires only minimal computational effort to calculate the control action, which is a big advantage over approaches that require online optimization, such as model predictive control.

Identifying the optimal design of a new launch vehicle is most important since design decisions made in the early development phase limit the vehicles' later performance and determines the associated costs. Reusing the first stage via retro-propulsive landing increases the complexity even more. Therefore, we develop an optimization framework for partially reusable launch vehicles, which enables multidisciplinary design studies. The framework contains suitable mass estimates of all essential subsystems and a routine to calculate the needed propellant for the ascent and landing maneuvers. For design optimization, the framework can be coupled with a genetic algorithm. The overall goal is to reveal the implications of different propellant combinations and objective functions on the launcher's optimal design for various mission scenarios. The results show that the optimization objective influences the most suitable propellant choice and the overall launcher design, concerning staging, weight, size, and rocket engine parameters. In terms of gross lift-off weight, liquid hydrogen seems to be favorable. When optimizing for a minimum structural mass or an expandable structural mass, hydrocarbon-based solutions show better results. Finally, launch vehicles using a hydrocarbon fuel in the first stage and liquid hydrogen in the upper stage are an appealing alternative, combining both fuels' benefits.

Methane is considered being a good choice as a propellant for future reusable launch systems. However, the heat transfer prediction for supercritical methane flowing in cooling channels of a regeneratively cooled combustion chamber is challenging. Because accurate heat transfer predictions are essential to design reliable and efficient cooling systems, heat transfer modeling is a fundamental issue to address. Advanced computational fluid dynamics (CFD) calculations achieve sufficient accuracy, but the associated computational cost prevents an efficient integration in optimization loops. Surrogate models based on artificial neural networks (ANNs) offer a great speed advantage. It is shown that an ANN, trained on data extracted from samples of CFD simulations, is able to predict the maximum wall temperature along straight rocket engine cooling channels using methane with convincing precision. The combination of the ANN model with simple relations for pressure drop and enthalpy rise results in a complete reduced order model, which can be used for numerically efficient design space exploration and optimization.

Fatigue life prediction is an essential part of multidisciplinary design studies and optimization loops, but state of the art finite element based methods are numerically inefficient. We overcome this challenge by training an artificial neural network to predict the number of cycles to failure, based on combustion chamber geometry and operational point. To accomplish this, a 2-d finite element analysis generates 250 000 training data samples. The trained network then predicts previously unseen data with a mean absolute percentage error of 6:8 % in less than 0:1 ms per sample compared to up to 5 min with finite element based methods. To the best of our knowledge, this publication is the first to successfully apply machine learning to fatigue life prediction.