A Comparison of Different State Representations for Reinforcement Learning Based Variable Speed Limit Control

izvorni znanstveni rad

izvorni znanstveni rad

A Comparison of Different State Representations for Reinforcement Learning Based Variable Speed Limit Control

Edouard Ivanjko
Vrsta prilog sa skupa (u zborniku)
Tip izvorni znanstveni rad
Godina 2018
Nadređena publikacija Proceedings of MED-2018
Stranice str. 266-271
DOI 10.1109/MED.2018.8442986
Status objavljeno

Sažetak

Variable Speed Limit Control (VSLC) is one control method for alleviating congestions on urban motorways. Machine learning techniques, like Reinforcement Learning (RL), are a promising alternative for setting up VSLC because an optimal control policy can be achieved with a smaller computational burden in comparison with optimal control approaches. A drawback is a large number of learning iterations and the problem of the exponential expansion of the state space dimension. This can be solved with function approximation techniques. Three different approaches for feature-based state representation in RL based VSLC are compared in this paper regarding the convergence of Total Time Spent. The microscopic traffic simulator VISSIM with a representative traffic model is used to evaluate the compared approaches. Results show that function approximation methods outperform RL based VSLC formulated with a lookup table by an average improvement of 10 %, where feature extraction methods (Coarse and Tile) coding showed slightly faster learning rate.

Ključne riječi

Intelligent transportation systems, Intelligent control systems, Variable speed limit control, Reinforcement learning, Tile coding, Coarse coding, RBF

Povezano

Ostale publikacije