Institute Homepage
Institute Homepage DE Sign In

Back

Movement Generation and Control Conference Paper 2020

Curious ilqr: Resolving uncertainty in model-based rl

Thumb ticker sm foto
Movement Generation and Control
Sarah Bechtle
Ph.D. Student
Thumb ticker sm akshara
Autonomous Motion
Akshara Rai
Thumb ticker sm righetti
Movement Generation and Control
Ludovic Righetti
Visiting Researcher

Curiosity as a means to explore during reinforcement learning problems has recently become very popular. However, very little progress has been made in utilizing curiosity for learning control. In this work, we propose a model-based reinforcement learning (MBRL) framework that combines Bayesian modeling of the system dynamics with curious iLQR, an iterative LQR approach that considers model uncertainty. During trajectory optimization the curious iLQR attempts to minimize both the task-dependent cost and the uncertainty in the dynamics model. We demonstrate the approach on reaching tasks with 7-DoF manipulators in simulation and on a real robot. Our experiments show that MBRL with curious iLQR reaches desired end-effector targets more reliably and with less system rollouts when learning a new task from scratch, and that the learned model generalizes better to new reaching tasks.

Author(s): Sarah Bechtle and Yixin Lin and Akshara Rai and Ludovic Righetti and Franziska Meier
Year: 2020
Month: May
Bibtex Type: Conference Paper (conference)
Event Name: Conference on Robot Learning
State: Published
Digital: True
Electronic Archiving: grant_archive

BibTex 

@conference{Bechtle2020curious,
  title = {Curious ilqr: Resolving uncertainty in model-based rl},
  abstract = {Curiosity as a means to explore during reinforcement learning problems has recently become very popular. However, very little progress has been made in utilizing curiosity for learning control. In this work, we propose a model-based reinforcement learning (MBRL) framework that combines Bayesian modeling of the system dynamics with curious iLQR, an iterative LQR approach that considers model uncertainty. During trajectory optimization the curious iLQR attempts to minimize both the task-dependent cost and the uncertainty in the dynamics model. We demonstrate the approach on reaching tasks with 7-DoF manipulators in simulation and on a real robot. Our experiments show that MBRL with curious iLQR reaches desired end-effector targets more reliably and with less system rollouts when learning a new task from scratch, and that the learned model generalizes better to new reaching tasks.},
  month = may,
  year = {2020},
  slug = {bechtle2020curious},
  author = {Bechtle, Sarah and Lin, Yixin and Rai, Akshara and Righetti, Ludovic and Meier, Franziska},
  month_numeric = {5}
}