Traffic Signal Control with Deep Reinforcement Learning and Self-attention Mechanism

Traffic signal control (TSC) is still one of the most important research topics in the transportation field. The existing traffic signal control method based on deep reinforcement learning (DRL) needs to be designed manually, and it is often difficult to extract the complete traffic state information in the real operations. This paper proposes a DRL algorithm based on the self-attention network for the traffic signal control to analyze the potential traffic from limited traffic state information and reduce the difficulty of traffic state design. The vehicle position of each entry lane at the intersection is obtained, and the vehicle position distribution matrix is established through the non-uniform quantization and one-hot encoding method. The self-attention network is then used to analyze the spatial correlation and latent information of the vehicle location distribution matrix which is an input of the DRL algorithm. The traffic signal adaptive control strategy is trained at a single intersection and the adaptability and robustness of the proposed algorithm are verified in a multi-intersection road network. The simulation results show that in a single intersection environment, the proposed algorithm has better performance on the average vehicle delay and other indicators compared with three benchmark algorithms. The proposed algorithm also has good adaptability in the multi-intersection road network.