The Library
Assessment of reward functions in reinforcement learning for multi-modal urban traffic control under real-world limitations
Tools
Tools
Tools
Egea, Alvaro Cabrejas and Connaughton, Colm (2021) Assessment of reward functions in reinforcement learning for multi-modal urban traffic control under real-world limitations. In: International Conference on Intelligent Transportation, Indianapolis, IN, USA, 19-22 Sep 2021. Published in: 2021 IEEE International Intelligent Transportation Systems Conference (ITSC) pp. 2095-2102. ISBN 9781728191423. doi:10.1109/ITSC48978.2021.9565054
|
PDF
WRAP-Assessment-reward-functions-reinforcement-learning-multi-modal-urban-traffic-control-real-world-2021.pdf - Accepted Version - Requires a PDF viewer. Download (909Kb) | Preview |
Official URL: http://dx.doi.org/10.1109/ITSC48978.2021.9565054
Abstract
Reinforcement Learning is proving a successful tool that can manage urban intersections with a fraction of the effort required to curate traditional traffic controllers. However, literature on the simultaneous introduction and control of pedestrians to such intersections is very scarce. Furthermore, it is unclear what traffic state variables should be used as reward to minimise waiting times. This paper robustly evaluates 30 different Reinforcement Learning reward functions for controlling a real-world intersection serving pedestrians and vehicles covering the main traffic state variables available via modern vision-based sensors. Some rewards proposed in previous literature solely for vehicular traffic are extended to pedestrians while new ones are proposed. We use a calibrated model in terms of demand, sensors, green times and other operational constraints of a real intersection in Greater Manchester, UK, to which the agent has been since deployed. The assessed rewards can be classified in 5 groups depending on the quantities used: queues, waiting time, delay, average speed and throughput in the junction. The performance of different agents, in terms of waiting time, is compared across different demand levels, from normal operation to saturation of traditional adaptive controllers. We find that those rewards maximising the speed of the network obtain the lowest waiting time for vehicles and pedestrians simultaneously, closely followed by queue minimisation, demonstrating better performance than other previously proposed methods.
| Item Type: | Conference Item (Paper) | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Subjects: | H Social Sciences > HE Transportation and Communications Q Science > QA Mathematics |
|||||||||
| Divisions: | Faculty of Science, Engineering and Medicine > Science > Mathematics | |||||||||
| Library of Congress Subject Headings (LCSH): | Reinforcement learning -- Mathematical models, Traffic engineering -- Mathematical models, City traffic, Traffic signs and signals -- Control systems | |||||||||
| Journal or Publication Title: | 2021 IEEE International Intelligent Transportation Systems Conference (ITSC) | |||||||||
| Publisher: | IEEE | |||||||||
| ISBN: | 9781728191423 | |||||||||
| Book Title: | 2021 IEEE International Intelligent Transportation Systems Conference (ITSC) | |||||||||
| Official Date: | 25 October 2021 | |||||||||
| Dates: |
|
|||||||||
| Page Range: | pp. 2095-2102 | |||||||||
| DOI: | 10.1109/ITSC48978.2021.9565054 | |||||||||
| Status: | Peer Reviewed | |||||||||
| Publication Status: | Published | |||||||||
| Reuse Statement (publisher, data, author rights): | © 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. | |||||||||
| Access rights to Published version: | Restricted or Subscription Access | |||||||||
| RIOXX Funder/Project Grant: |
|
|||||||||
| Conference Paper Type: | Paper | |||||||||
| Title of Event: | International Conference on Intelligent Transportation | |||||||||
| Type of Event: | Conference | |||||||||
| Location of Event: | Indianapolis, IN, USA | |||||||||
| Date(s) of Event: | 19-22 Sep 2021 |
Request changes or add full text files to a record
Repository staff actions (login required)
 |
View Item |
Downloads
Downloads per month over past year
