Process control is essential in advanced manufacturing to minimize variations caused by disturbances from manufacturing environments. As precision requirements continue to rise, process models have become increasingly complex, making it difficult for model-based controllers to accurately characterize the relationship between control inputs and system outputs. Meanwhile, unstable and unobservable disturbances, which are inevitable during manufacturing processes, introduce significant variations for model-free controllers. To address this challenge, this work develops a novel model-free reinforcement learning (RL) control approach that leverages model-based prior knowledge. Although prior knowledge cannot define the process model, it helps simplify the model structure and reduce variations in model-free controllers. Unlike traditional RL methods that require extensive real-time experimentation, we introduce an offline RL approach that consists of offline learning and online control phases and relies solely on fixed historical data. During the offline learning phase, nonparametric regression is employed to approximate the unknown process, while Gaussian process is used to predict disturbances. These two methods are integrated with a policy-gradient-search RL algorithm in online control. Theoretical properties of our control methodology are discussed. Simulations and a crystallization case study are conducted to validate the effectiveness of the proposed method.
Run-to-run (R2R) control is widely used in advanced manufacturing. Conventional R2R control methods often assume that system disturbances exhibit linear structures or low-rank approximations in high-dimensional data, such as images or videos. While these assumptions facilitate implementation, they limit the effectiveness of R2R control in managing sophisticated manufacturing processes with intricate nonlinear disturbances. To address these challenges, we propose a novel nonlinear spatio-temporal R2R control framework that estimates system disturbances using a diffusion model and applies control actions to compensate for them. The proposed approach integrates an offline modeling phase to capture disturbance dynamics and an online control phase that dynamically adjusts control actions to minimize deviations from the target system response. Unlike conventional deterministic control strategies, the proposed method explicitly quantifies uncertainty, enabling more robust and informed decision-making. The effectiveness of this method is validated through simulation studies and a case study, demonstrating its adaptability in complex, high-dimensional environments.
