Application of Markov renewal theory and semi‐Markov decision processes in maintenance modeling and optimization of multi‐unit systems

Abstract In this paper, a condition‐based maintenance model for a multi‐unit production system is proposed and analyzed using Markov renewal theory. The units of the system are subject to gradual deterioration, and the gradual deterioration process of each unit is described by a three‐state continuous time homogeneous Markov chain with two working states and a failure state. The production rate of the system is influenced by the deterioration process and the demand is constant. The states of the units are observable through regular inspections and the decision to perform maintenance depends on the number of units in each state. The objective is to obtain the steady‐state characteristics and the formula for the long‐run average cost for the controlled system. The optimal policy is obtained using a dynamic programming algorithm. The result is validated using a semi‐Markov decision process formulation and the policy iteration algorithm. Moreover, an analytical expression is obtained for the calculation of the mean time to initiate maintenance using the first passage time theory.