Empirical Inference Poster 2003

A Representation of Complex Movement Sequences Based on Hierarchical Spatio-Temporal Correspondence for Imitation Learning in Robotics

Imitation learning of complex movements has become a popular topic in neuroscience, as well as in robotics. A number of conceptual as well as practical problems are still unsolved. One example is the determination of the aspects of movements which are relevant for imitation. Problems concerning the movement representation are twofold: (1) The movement characteristics of observed movements have to be transferred from the perceptual level to the level of generated actions. (2) Continuous spaces of movements with variable styles have to be approximated based on a limited number of learned example sequences. Therefore, one has to use representation with a high generalisation capability. We present methods for the representation of complex movement sequences that addresses these questions in the context of the imitation learning of writing movements using a robot arm with human-like geometry. For the transfer of complex movements from perception to action we exploit a learning-based method that represents complex action sequences by linear combination of prototypical examples (Ilg and Giese, BMCV 2002). The method of hierarchical spatio-temporal morphable models (HSTMM) decomposes action sequences automatically into movement primitives. These primitives are modeled by linear combinations of a small number of learned example trajectories. The learned spatio-temporal models are suitable for the analysis and synthesis of long action sequences, which consist of movement primitives with varying style parameters. The proposed method is illustrated by imitation learning of complex writing movements. Human trajectories were recorded using a commercial motion capture system (VICON). In the rst step the recorded writing sequences are decomposed into movement primitives. These movement primitives can be analyzed and changed in style by de ning linear combinations of prototypes with di erent linear weight combinations. Our system can imitate writing movements of di erent actors, synthesize new writing styles and can even exaggerate the writing movements of individual actors. Words and writing movements of the robot look very natural, and closely match the natural styles. These preliminary results makes the proposed method promising for further applications in learning-based robotics. In this poster we focus on the acquisition of the movement representation (identi cation and segmentation of movement primitives, generation of new writing styles by spatio-temporal morphing). The transfer of the generated writing movements to the robot considering the given kinematic and dynamic constraints is discussed in Bakir et al (this volume).

Author(s): Ilg, W. and Bakir, GH. and Franz, MO. and Giese, M.
Volume: 6
Pages: 74
Year: 2003
Month: February
Day: 0
Editors: H.H. Bülthoff, K.R. Gegenfurtner, H.A. Mallot, R. Ulrich, F.A. Wichmann
Bibtex Type: Poster (poster)
Digital: 0
Electronic Archiving: grant_archive
Event Name: 6. Tübinger Wahrnehmungskonferenz (TWK 2003)
Event Place: Tübingen, Germany
Organization: Max-Planck-Gesellschaft
School: Biologische Kybernetik
Links:

BibTex

@poster{2076,
  title = {A Representation of Complex Movement Sequences Based on Hierarchical Spatio-Temporal Correspondence for Imitation Learning in Robotics},
  abstract = {Imitation learning of complex movements has become a popular topic in neuroscience,
  as well as in robotics. A number of conceptual as well as practical problems are still
  unsolved. One example is the determination of the aspects of movements which are
  relevant for imitation. Problems concerning the movement representation are twofold:
  (1) The movement characteristics of observed movements have to be transferred from
  the perceptual level to the level of generated actions. (2) Continuous spaces of movements
  with variable styles have to be approximated based on a limited number of
  learned example sequences. Therefore, one has to use representation with a high generalisation
  capability. We present methods for the representation of complex movement
  sequences that addresses these questions in the context of the imitation learning of
  writing movements using a robot arm with human-like geometry. For the transfer of
  complex movements from perception to action we exploit a learning-based method that
  represents complex action sequences by linear combination of prototypical examples (Ilg
  and Giese, BMCV 2002). The method of hierarchical spatio-temporal morphable models
  (HSTMM) decomposes action sequences automatically into movement primitives.
  These primitives are modeled by linear combinations of a small number of learned
  example trajectories. The learned spatio-temporal models are suitable for the analysis
  and synthesis of long action sequences, which consist of movement primitives with
  varying style parameters. The proposed method is illustrated by imitation learning
  of complex writing movements. Human trajectories were recorded using a commercial
  motion capture system (VICON). In the rst step the recorded writing sequences
  are decomposed into movement primitives. These movement primitives can be analyzed
  and changed in style by dening linear combinations of prototypes with dierent
  linear weight combinations. Our system can imitate writing movements of dierent
  actors, synthesize new writing styles and can even exaggerate the writing movements
  of individual actors. Words and writing movements of the robot look very natural, and
  closely match the natural styles. These preliminary results makes the proposed method
  promising for further applications in learning-based robotics. In this poster we focus
  on the acquisition of the movement representation (identication and segmentation of
  movement primitives, generation of new writing styles by spatio-temporal morphing).
  The transfer of the generated writing movements to the robot considering the given
  kinematic and dynamic constraints is discussed in Bakir et al (this volume).},
  volume = {6},
  pages = {74},
  editors = {H.H. Bülthoff, K.R. Gegenfurtner, H.A. Mallot, R. Ulrich, F.A. Wichmann},
  organization = {Max-Planck-Gesellschaft},
  school = {Biologische Kybernetik},
  month = feb,
  year = {2003},
  slug = {2076},
  author = {Ilg, W. and Bakir, GH. and Franz, MO. and Giese, M.},
  month_numeric = {2}
}