Hierarchical Energy Management Recognizing Powertrain Dynamics for Electrified Vehicles With Deep Reinforcement Learning and Transfer Learning

Deep reinforcement learning (DRL)-based energy management strategies (EMSs) have gained significant popularity in improving the performance of electrified vehicles. Typically, these EMSs are trained and validated in simulated environments. However, this article reveals that the environment’s fidelity significantly impacts DRL-based EMSs’ performance. Specifically, the EMSs optimized within low-fidelity environments (LFEs)—prevalent in literature yet lacking detailed powertrain dynamics—suffer a performance drop of 2%–3% in energy economy when tested in high-fidelity environments (HFEs) that incorporate powertrain dynamics. To address it, a DRL-based hierarchical energy management framework for multimode power-split hybrid electric vehicles (HEVs) is proposed. It recognizes powertrain dynamics and facilitates transfer learning techniques to bridge the performance gap between LFEs and HFEs. In the upper level of this framework, a DRL agent determines the optimal timing to activate the hybrid mode and the optimal engine operation. The lower level optimizes the torque distribution between two electric motors for all-electric modes. Simulation results demonstrate that the proposed DRL-based EMS, enhanced by transfer learning, reduces training time by approximately 40% compared with the trained-from-scratch EMS within an HFE. Moreover, the proposed EMS achieves 98% energy economy of the optimal benchmark, addressing the noted performance degradation and exhibiting consistent performance in adaptability tests.