DRL-Based Joint Resource Allocation and Platoon Control Optimization for UAV-Hosted Platoon Digital Twin

Digital twin (DT)-empowered platoon can improve platoon management efficiency and driving safety. However, the resource allocation scheme of low-latency platoon DT (PDT) and the interactions with platoon control strategy are important issues in the study of PDTs. In this article, we study the resource allocation in the PDT network and the interaction mechanism between PDT and platoon control for an unmanned aerial vehicle (UAV)-hosted PDT. We introduce the Age of Information (AoI) metrics to characterize the freshness of the DTs. To explore the impact of the PDT resource allocation scheme on the platoon control strategy, we propose a joint optimization model for power resource allocation and platoon control. Specifically, the allocation of power resources affects the PDT’s AoI, and the high-latency PDT in turn affects the platoon control strategy. Our objective is minimize the weighted sum of the system’s average energy consumption and the PDT’s average peak AoI. To solve the problem, we first reformulate the power resource allocation problem over a period of time as a Markov decision process (MDP) model, and then propose the Dirichlet deep deterministic policy gradient (DDPG)-based power allocation (D3PGPA) method based on Dirichlet distribution and DDPG algorithm. The method can not only effectively explores the state space while satisfying the constraints of limited resources but also improve the stability of the algorithm. Numerical results show that the D3PGPA method can host a PDT with low AoI and improve the stability of the platoon. Besides, our proposed method performs stably and outperforms other benchmark methods.