Optimum Digital Twin Response Time for Time-Sensitive Applications

As the digital replica of a physical system (PS), a digital twin (DT) is responsible for providing real-time information of its PS to applications. However, random network conditions result in uncertainty in future age of information (AoI) at the DT, which makes it complicated for a DT to decide when to response an application request in order to maintain the best information freshness at the application. In this work, we consider the effect of random wireless channel condition between the PS and the DT on the AoI changes at the DT, and formulate a Markov decision process that finds the optimum response time for the DT to send the PS information to an application after receiving a request from the application. The objective is to minimize the average AoI at the application. The MDP has delayed reward, and is solved by redistributing the reward with LSTM network and then finding the optimal policies using Dueling Double Deep Q-learning. Numerical results show that the solutions provide close-to-optimum average AoI performance.