Condition-based maintenance (CBM) and spare provisioning are both important to guarantee the operation of redundant systems composed of degrading components. However, most existing studies on joint optimization of CBM and spare inventory assume maintenance actions are instantaneous and the lead time for spares are fixed, which are not consistent with the reality. Therefore, this paper focus on the joint optimization problems to minimize the total cost rate considering stochastic maintenance time for components and stochastic lead time for spares. The problem is modeled as a Markov decision process model and solved by an improved reinforcement learning algorithm, i.e., the improved Q-learning algorithm, which converges more quickly and reaches a smaller value of the total cost rate than the traditional Q-learning algorithm. Moreover, the simulated environment based on discrete event simulation method is introduced in detail and the convergence of the algorithm is proved theoretically. Based on the numerical study, we further demonstrate the convergence and effectiveness of the proposed algorithm and perform sensitivity analysis on several model parameters to provide management insights for decision makers.

Condition‐based maintenance has been studied for managing the degradation of components in a system. However, components in different positions may undergo different workload, leading to imbalanced components degradation. Component reallocation is an effective method to balance the degradation of components in different positions with different degradation rates. Thus, this paper investigates the joint optimization of condition‐based maintenance and condition‐based reallocation for a non‐repairable system with multiple identical and functionally‐exchangeable components that undergo different continuous degradation processes due to different workload. We model this problem through Markov decision process, which is solved by a dynamic programming algorithm, and the objective is to provide the optimal maintenance and reallocation actions exactly for each system state, that is, the joint optimal policy, to minimize the discounted long‐run total cost. Especially, our joint policy is on component‐level and proved to be optimal. Finally, the numerical analysis shows the structural insights and effectiveness of the proposed joint policy, for which the sensitivity analysis on the degradation parameters, setup cost, reallocation cost, and downtime cost is performed. We also propose a new reallocation strategy and show the results and computation time for systems with different scales, which indicates that our model can handle systems with reasonable size.

The components in a system undergo different workload due to being in different positions, resulting in unbalanced component degradation level. In order to balance the degradation and extend the lifetime of the system, component reallocation is a feasible way to achieve degradation balance in some cases that we cannot manage the operator's behavior to control components' workload. In this paper, we incorporate component reallocation into condition-based maintenance for a non-repairable series system consisting of two components. The degradation processes of components are assumed to follow the Wiener process with different degradation parameters. The problem is formulated as a Markov decision process and solved by the value iteration algorithm to obtain the optimal replacement and reallocation policy that minimizes the discounted system long-term total cost. A numerical example gives the optimal inspection interval and the properties of the optimal policy. The sensitivity analysis with respect to reallocation cost is provided to illustrate the optimal replacement and reallocation policy.