A Practical Reinforcement Learning (RL) Controller Design for Nonlinear Systems

This paper presents a practically implementable reinforcement learning (RL) approach for process control applications. Standard model-free RL approaches are not applicable in practice because the learning process of an RL agent requires random exploration of state and action spaces, which can compromise process safety and economic objectives. To tackle this issue, an offline training strategy is proposed by leveraging existing model predictive control (MPC) to pre-train the RL agent. MPC actions, calculated offline by solving the MPC optimization problem based on a wide range of initial conditions, along with its objective function are utilized to pretrain the actor and critics of the RL agent. The pre-trained RL controller, with similar performance to the MPC performance, is then utilized for online control for further training. The efficacy of the proposed RL controller to improve the tracking performance is demonstrated using a simulation example for a pH neutralization process.