A knowledge-driven deep reinforcement learning approach for dynamic scheduling of re-entrant hybrid flow shop with in-line product quality inspection

Real-time product quality inspection (QI) of products during manufacturing enables early defect detection and reduces resource waste. However, dynamic disturbances arising from QI, such as machine maintenance and rework operations, impose significant challenges to real-time production scheduling. In this study, a dynamic re-entrant hybrid flow shop scheduling problem considering in-line product quality inspection (DHFSP-QI) is investigated, where product quality variations dynamically impact production scheduling. To address this challenge, a multi-agent system (MAS) is developed to model dynamic shop-floor interactions among machines, quality detectors, products, and scheduling units. A knowledge-driven deep reinforcement learning (DRL) framework integrated with variable neighborhood search (KDRL-VNS) is proposed. The VNS-enhanced reward feedback mechanism guides agents to acquire efficient strategies. The DRL-enhanced scheduling agents use graph attention networks (GATs) to extract graph-based state representations of the workshop in real time, thereby enabling dynamic-scheduling decisions aimed at the minimizing total weighted tardiness. Experimental evaluations across multiple scenarios demonstrate that the proposed method, by incorporating VNS-based expert knowledge, outperforms various heuristic algorithms, genetic programming algorithms and DRL algorithms. It achieves accelerated convergence and delivers an average 4.6% relative improvement in performance compared to DRL methods.