Perceiving Systems Conference Paper 2019

Learning to Train with Synthetic Humans

Lala2

Neural networks need big annotated datasets for training. However, manual annotation can be too expensive or even unfeasible for certain tasks, like multi-person 2D pose estimation with severe occlusions. A remedy for this is synthetic data with perfect ground truth. Here we explore two variations of synthetic data for this challenging problem; a dataset with purely synthetic humans, as well as a real dataset augmented with synthetic humans. We then study which approach better generalizes to real data, as well as the influence of virtual humans in the training loss. We observe that not all synthetic samples are equally informative for training, while the informative samples are different for each training stage. To exploit this observation, we employ an adversarial student-teacher framework; the teacher improves the student by providing the hardest samples for its current state as a challenge. Experiments show that this student-teacher framework outperforms all our baselines.

Author(s): David T. Hoffmann and Dimitrios Tzionas and Michael J. Black and Siyu Tang
Book Title: German Conference on Pattern Recognition (GCPR)
Pages: 609--623
Year: 2019
Month: September
Publisher: Springer International Publishing
Project(s):
Bibtex Type: Conference Paper (inproceedings)
DOI: https://doi.org/10.1007/978-3-030-33676-9_43
URL: https://ltsh.is.tue.mpg.de
Electronic Archiving: grant_archive
ISBN: 978-3-030-33676-9
Attachments:

BibTex

@inproceedings{Hoffmann:GCPR:2019,
  title = {Learning to Train with Synthetic Humans},
  booktitle = {German Conference on Pattern Recognition (GCPR)},
  abstract = {Neural networks need big annotated datasets for training. However, manual annotation can be too expensive or even unfeasible for certain tasks, like multi-person 2D pose estimation with severe occlusions. A remedy for this is synthetic data with perfect ground truth. Here we explore two variations of synthetic data for this challenging problem; a dataset with purely synthetic humans, as well as a real dataset augmented with synthetic humans. We then study which approach better generalizes to real data, as well as the influence of virtual humans in the training loss. We observe that not all synthetic samples are equally informative for training, while the informative samples are different for each training stage. To exploit this observation, we employ an adversarial student-teacher framework; the teacher improves the student by providing the hardest samples for its current state as a challenge. Experiments show that this student-teacher framework outperforms all our baselines.},
  pages = {609--623},
  publisher = {Springer International Publishing},
  month = sep,
  year = {2019},
  slug = {hoffmann-gcpr-2019},
  author = {Hoffmann, David T. and Tzionas, Dimitrios and Black, Michael J. and Tang, Siyu},
  url = {https://ltsh.is.tue.mpg.de},
  month_numeric = {9}
}