Abstract

Inspection and maintenance activities are effective ways to reveal and restore the health conditions of many industrial systems, respectively. Most extant works on inspection and maintenance optimization problems assumed that systems operate under a time-invariant demand. Such a simplified assumption is oftentimes violated by a changeable market environment, seasonal factors, and even unexpected emergencies. In this article, with the aim of minimizing the expected total cost associated with inspections, maintenance, and unsupplied demand, a dynamic inspection and maintenance scheduling model is put forth for multi-state systems (MSSs) under a time-varying demand. Non-periodic inspections are performed on the components of an MSS and imperfect maintenance actions are dynamically scheduled based on the inspection results. By introducing the concept of decision epochs, the resulting inspection and maintenance scheduling problem is formulated as a Markov decision process (MDP). The deep reinforcement learning (DRL) method with a proximal policy optimization (PPO) algorithm is customized to cope with the “curse of dimensionality” of the resulting sequential decision problem. As an extra input feature for the agent, the category of decision epochs is formulated to improve the effectiveness of the customized DRL method. A six-component MSS, along with a multi-state coal transportation system, is given to demonstrate the effectiveness of the proposed method.

摘要

检查和维护活动分别是揭示和恢复许多工业系统健康状况的有效方法。大多数现有的检查和维护优化问题工作都假设系统在时不变的需求下运行。这种简化的假设常常会被多变的市场环境、季节性因素甚至意外的紧急情况所打破。在本文中，为了最小化与检查、维护和未供应需求相关的预期总成本，针对时变需求下的多状态系统（MSS）提出了动态检查和维护调度模型。对 MSS 的组件进行不定期检查，并根据检查结果动态安排不完善的维护操作。通过引入决策时期的概念，所产生的检查和维护调度问题被表述为马尔可夫决策过程（MDP）。具有近端策略优化（PPO）算法的深度强化学习（DRL）方法经过定制，可以应对由此产生的顺序决策问题的“维数灾难”。作为智能体的额外输入特征，制定了决策时期的类别，以提高定制 DRL 方法的有效性。给出了六分量 MSS 以及多状态煤炭运输系统，以证明该方法的有效性。具有近端策略优化（PPO）算法的深度强化学习（DRL）方法经过定制，可以应对由此产生的顺序决策问题的“维数灾难”。作为智能体的额外输入特征，制定了决策时期的类别，以提高定制 DRL 方法的有效性。给出了六分量 MSS 以及多状态煤炭运输系统，以证明该方法的有效性。具有近端策略优化（PPO）算法的深度强化学习（DRL）方法经过定制，可以应对由此产生的顺序决策问题的“维数灾难”。作为智能体的额外输入特征，制定了决策时期的类别，以提高定制 DRL 方法的有效性。给出了六分量 MSS 以及多状态煤炭运输系统，以证明该方法的有效性。