Empirical Inference Conference Paper 2011

Reinforcement Learning to adjust Robot Movements to New Situations

Many complex robot motor skills can be represented using elementary movements, and there exist efficient techniques for learning parametrized motor plans using demonstrations and self-improvement. However, in many cases, the robot currently needs to learn a new elementary movement even if a parametrized motor plan exists that covers a similar, related situation. Clearly, a method is needed that modulates the elementary movement through the meta-parameters of its representation. In this paper, we show how to learn such mappings from circumstances to meta-parameters using reinforcement learning.We introduce an appropriate reinforcement learning algorithm based on a kernelized version of the reward-weighted regression. We compare this algorithm to several previous methods on a toy example and show that it performs well in comparison to standard algorithms. Subsequently, we show two robot applications of the presented setup; i.e., the generalization of throwing movements in darts, and of hitting movements in table tennis. We show that both tasks can be learned successfully using simulated and real robots.

Author(s): Kober, J. and Oztop, E. and Peters, J.
Book Title: Robotics: Science and Systems VI
Journal: Robotics: Science and Systems VI
Pages: 33-40
Year: 2011
Month: September
Day: 0
Editors: Matsuoka, Y. , H. F. Durrant-Whyte, J. Neira
Publisher: MIT Press
Bibtex Type: Conference Paper (inproceedings)
Address: Cambridge, MA, USA
Event Name: 2010 Robotics: Science and Systems Conference (RSS 2010)
Event Place: Zaragoza, Spain
Digital: 0
Electronic Archiving: grant_archive
ISBN: 978-0-262-51681-5
Language: en
Organization: Max-Planck-Gesellschaft
School: Biologische Kybernetik
Links:

BibTex

@inproceedings{6438,
  title = {Reinforcement Learning to adjust Robot Movements to New Situations},
  journal = {Robotics: Science and Systems VI},
  booktitle = {Robotics: Science and Systems VI},
  abstract = {Many complex robot motor skills can be represented using elementary movements, and there exist efficient
  techniques for learning parametrized motor plans using demonstrations and self-improvement. However, in
  many cases, the robot currently needs to learn a new elementary movement even if a parametrized motor
  plan exists that covers a similar, related situation. Clearly, a method is needed that modulates the elementary
  movement through the meta-parameters of its representation. In this paper, we show how to learn such
  mappings from circumstances to meta-parameters using reinforcement learning.We introduce an appropriate
  reinforcement learning algorithm based on a kernelized version of the reward-weighted regression. We
  compare this algorithm to several previous methods on a toy example and show that it performs well in
  comparison to standard algorithms. Subsequently, we show two robot applications of the presented setup;
  i.e., the generalization of throwing movements in darts, and of hitting movements in table tennis. We show
  that both tasks can be learned successfully using simulated and real robots.},
  pages = {33-40},
  editors = {Matsuoka, Y. , H. F. Durrant-Whyte, J. Neira},
  publisher = {MIT Press},
  organization = {Max-Planck-Gesellschaft},
  school = {Biologische Kybernetik},
  address = {Cambridge, MA, USA},
  month = sep,
  year = {2011},
  slug = {6438},
  author = {Kober, J. and Oztop, E. and Peters, J.},
  month_numeric = {9}
}