In this paper, we introduce a simulation testbed framework to evaluate the performance of a self-learning adaptive traffic signal control system. The core contribution of this paper is the assessment of the system’s two modes of operations (independent versus coordinated) under different congestion levels and network configurations. The insights and conclusions of the paper are based on the synergetic effect of the following: (1) appropriate design of the adaptive system parameters, (2) seamless design of generic interfaces between the adaptive system and the simulation environment using application programming interfaces, (3) rigorously calibrated simulation model and a comprehensive set of performance and environmental measures, and (4) investigation of the system components required for building a complete functioning system in the field. The system was designed and lab-tested on two case studies in the City of Burlington, Ontario. The intersections were designed and operated using the adaptive system and compared to the actuated optimized and coordinated base case timings plans. The analysis of the simulation results shows that overall the adaptive system outperforms the base case scenario by up to 25% savings in delay at the network level, and 15% reduction in CO2 emission. On the other hand, the results of the two testbed models indicate that the performance of the adaptive system varies according to the intersection conditions and flows, network configuration, traffic volume, variability in flow arrivals, and the proximity of intersections to each other.