Autonomous Motion Article 2010

Learning control in robotics – trajectory-based opitimal control techniques

In a not too distant future, robots will be a natural part of daily life in human society, providing assistance in many areas ranging from clinical applications, education and care giving, to normal household environments [1]. It is hard to imagine that all possible tasks can be preprogrammed in such robots. Robots need to be able to learn, either by themselves or with the help of human supervision. Additionally, wear and tear on robots in daily use needs to be automatically compensated for, which requires a form of continuous self-calibration, another form of learning. Finally, robots need to react to stochastic and dynamic environments, i.e., they need to learn how to optimally adapt to uncertainty and unforeseen changes. Robot learning is going to be a key ingredient for the future of autonomous robots. While robot learning covers a rather large field, from learning to perceive, to plan, to make decisions, etc., we will focus this review on topics of learning control, in particular, as it is concerned with learning control in simulated or actual physical robots. In general, learning control refers to the process of acquiring a control strategy for a particular control system and a particular task by trial and error. Learning control is usually distinguished from adaptive control [2] in that the learning system can have rather general optimization objectivesâ??not just, e.g., minimal tracking errorâ??and is permitted to fail during the process of learning, while adaptive control emphasizes fast convergence without failure. Thus, learning control resembles the way that humans and animals acquire new movement strategies, while adaptive control is a special case of learning control that fulfills stringent performance constraints, e.g., as needed in life-critical systems like airplanes. Learning control has been an active topic of research for at least three decades. However, given the lack of working robots that actually use learning components, more work needs to be done before robot learning will make it beyond the laboratory environment. This article will survey some ongoing and past activities in robot learning to assess where the field stands and where it is going. We will largely focus on nonwheeled robots and less on topics of state estimation, as typically explored in wheeled robots [3]â??6], and we emphasize learning in continuous state-action spaces rather than discrete state-action spaces [7], [8]. We will illustrate the different topics of robot learning with examples from our own research with anthropomorphic and humanoid robots.

Author(s): Schaal, S. and Atkeson, C. G.
Book Title: Robotics and Automation Magazine
Volume: 17
Number (issue): 2
Pages: 20-29
Year: 2010
Bibtex Type: Article (article)
URL: http://www-clmc.usc.edu/publications/S/schaal-RAM2010.pdf
Cross Ref: p10415
Electronic Archiving: grant_archive
Note: clmc

BibTex

@article{Schaal_RAM_2010,
  title = {Learning control in robotics -- trajectory-based opitimal control techniques},
  booktitle = {Robotics and Automation Magazine},
  abstract = {In a not too distant future, robots will be a natural part of
  daily life in human society, providing assistance in many
  areas ranging from clinical applications, education and care
  giving, to normal household environments [1]. It is hard to
  imagine that all possible tasks can be preprogrammed in such
  robots. Robots need to be able to learn, either by themselves
  or with the help of human supervision. Additionally, wear and
  tear on robots in daily use needs to be automatically compensated
  for, which requires a form of continuous self-calibration,
  another form of learning. Finally, robots need to react to stochastic
  and dynamic environments, i.e., they need to learn
  how to optimally adapt to uncertainty and unforeseen
  changes. Robot learning is going to be a key ingredient for the
  future of autonomous robots.
  While robot learning covers a rather large field, from learning
  to perceive, to plan, to make decisions, etc., we will focus
  this review on topics of learning control, in particular, as it is
  concerned with learning control in simulated or actual physical
  robots. In general, learning control refers to the process of
  acquiring a control strategy for a particular control system and
  a particular task by trial and error. Learning control is usually
  distinguished from adaptive control [2] in that the learning system
  can have rather general optimization objectivesâ??not just,
  e.g., minimal tracking errorâ??and is permitted to fail during
  the process of learning, while adaptive control emphasizes fast
  convergence without failure. Thus, learning control resembles
  the way that humans and animals acquire new movement
  strategies, while adaptive control is a special case of learning
  control that fulfills stringent performance constraints, e.g., as
  needed in life-critical systems like airplanes.
  Learning control has been an active topic of research for at
  least three decades. However, given the lack of working robots
  that actually use learning components, more work needs to be
  done before robot learning will make it beyond the laboratory
  environment. This article will survey some ongoing and past
  activities in robot learning to assess where the field stands and
  where it is going. We will largely focus on nonwheeled robots
  and less on topics of state estimation, as typically explored in
  wheeled robots [3]â??6], and we emphasize learning in continuous
  state-action spaces rather than discrete state-action spaces [7], [8].
  We will illustrate the different topics of robot learning with
  examples from our own research with anthropomorphic and
  humanoid robots.},
  volume = {17},
  number = {2},
  pages = {20-29},
  year = {2010},
  note = {clmc},
  slug = {schaal_ram_2010},
  author = {Schaal, S. and Atkeson, C. G.},
  crossref = {p10415},
  url = {http://www-clmc.usc.edu/publications/S/schaal-RAM2010.pdf}
}