Empirical Inference
Conference Paper
2008
Policy Gradients with Parameter-based Exploration for Control
We present a model-free reinforcement learning method for partially observable Markov decision problems. Our method estimates a likelihood gradient by sampling directly in parameter space, which leads to lower variance gradient estimates than those obtained by policy gradient methods such as REINFORCE. For several complex control tasks, including robust standing with a humanoid robot, we show that our method outperforms well-known algorithms from the fields of policy gradients, finite difference methods and population based heuristics. We also provide a detailed analysis of the differences between our method and the other algorithms.
Author(s): | Sehnke, F. and Osendorfer, C. and Rückstiess, T. and Graves, A. and Peters, J. and Schmidhuber, J. |
Book Title: | ICANN 2008 |
Journal: | Artificial Neural Networks: ICANN 2008 |
Pages: | 387-396 |
Year: | 2008 |
Month: | September |
Day: | 0 |
Editors: | Kurkova-Pohlova, V. , R. Neruda, J. Koutnik |
Publisher: | Springer |
Bibtex Type: | Conference Paper (inproceedings) |
Address: | Berlin, Germany |
DOI: | 10.1007/978-3-540-87536-9_40 |
Event Name: | 18th International Conference on Artificial Neural Networks |
Event Place: | Praha, Czech Republic |
Digital: | 0 |
Electronic Archiving: | grant_archive |
Institution: | European Neural Network Society |
Language: | en |
Organization: | Max-Planck-Gesellschaft |
School: | Biologische Kybernetik |
Links: |
BibTex
@inproceedings{5169, title = {Policy Gradients with Parameter-based Exploration for Control}, journal = {Artificial Neural Networks: ICANN 2008}, booktitle = {ICANN 2008}, abstract = {We present a model-free reinforcement learning method for partially observable Markov decision problems. Our method estimates a likelihood gradient by sampling directly in parameter space, which leads to lower variance gradient estimates than those obtained by policy gradient methods such as REINFORCE. For several complex control tasks, including robust standing with a humanoid robot, we show that our method outperforms well-known algorithms from the fields of policy gradients, finite difference methods and population based heuristics. We also provide a detailed analysis of the differences between our method and the other algorithms.}, pages = {387-396}, editors = {Kurkova-Pohlova, V. , R. Neruda, J. Koutnik}, publisher = {Springer}, organization = {Max-Planck-Gesellschaft}, institution = {European Neural Network Society}, school = {Biologische Kybernetik}, address = {Berlin, Germany}, month = sep, year = {2008}, slug = {5169}, author = {Sehnke, F. and Osendorfer, C. and R{\"u}ckstiess, T. and Graves, A. and Peters, J. and Schmidhuber, J.}, month_numeric = {9} }