A Case for Monte Carlo Tree Search in Adaptive Traffic Signal Control: Modifiability, Interpretability and Generalization**The authors would like to thank Aimsun for providing research licenses for the Aimsun simulator

Adaptive Traffic Signal Control methods based on Reinforcement Learning have been applied successfully to reach state-of-the-art results in simulation. However, most recent works in the area use model-free methods, which learn value and/or policy functions directly through environment in-teractions without a dynamics model. Model-free methods have several potential drawbacks: (1) difficulties in generalization to dynamics not seen in training, as well as difficulties in transfer to different intersection geometries; (2) low sample-efficiency, resulting in high resource requirements for training; (3) often, reliance on difficult-to-interpret function approximators such as neural networks. In this work, we demonstrate experimentally that Monte Carlo Tree Search (MCTS) with a simple queue and platoon-propagation dynamics model can reach lower delay on a single intersection than the standard model-free Deep Q-Networks (DQN) algorithm. Moreover, we provide empirical results that illustrate that model-based receding-horizon planning can alleviate many of the drawbacks of the model-free methods listed above: better generalization characteristics; more direct and less resource-demanding transfer to different intersection geometries and phasing schemes; and simpler visualization and interpretation, aiding modification and debugging.