Deep Merging: Vehicle Merging Controller Based on Deep Reinforcement Learning with Embedding Network

Vehicles at highway merging sections must make lane changes to join the highway. This lane change can generate congestion. To reduce congestion, vehicles should merge so as not to affect traffic flow as much as possible. In our study, we propose a vehicle controller called Deep Merging that uses deep reinforcement learning to improve the merging efficiency of vehicles while considering the impact on traffic flow. The system uses the images of a merging section as input to output the target vehicle speed. Moreover, an embedding network for estimating the controlled vehicle speed is introduced to the deep reinforcement learning network architecture to improve the learning efficiency. In order to show the effectiveness of the proposed method, the merging behavior and traffic conditions in several situations are verified by experiments using a traffic simulator. Through these experiments, it is confirmed that the proposed method enables controlled vehicles to effectively merge without adversely affecting to the traffic flow.