A practical reinforcement learning control design for nonlinear systems with input and output constraints

In this work, a practically implementable reinforcement learning (RL)-based controller is designed to handle process input and output constraints. In a typical RL problem, an RL agent is employed to learn an optimal control policy through interactions with the environment. This is unimplementable in practical situations due to the excessive exploration needed by the RL-based controller and exacerbated by the possible violation of the input and output constraints. We previously proposed an implementable RL controller that can circumvent random exploration needs by leveraging existing model predictive control (MPC) to pre-train/warm start the RL agent. The pre-trained agent is subsequently employed in real-time to engage with the process to improve its performance by gaining more knowledge about the nonlinear behavior of the system. This work generalizes our previous method to handle constraints on the outputs and the rate of change of the inputs by modifying the reward function. The effectiveness of the proposed algorithm is illustrated through simulations conducted for control of a pH neutralization process. The findings indicate that the proposed RL method enhances closed-loop performance in comparison to the nominal MPC while satisfying all input and output constraints.