Perzeptive Systeme Conference Paper 2020

VIBE: Video Inference for Human Body Pose and Shape Estimation

Vibe

Human motion is fundamental to understanding behavior. Despite progress on single-image 3D pose and shape estimation, existing video-based state-of-the-art methodsfail to produce accurate and natural motion sequences due to a lack of ground-truth 3D motion data for training. To address this problem, we propose “Video Inference for Body Pose and Shape Estimation” (VIBE), which makes use of an existing large-scale motion capture dataset (AMASS) together with unpaired, in-the-wild, 2D keypoint annotations. Our key novelty is an adversarial learning framework that leverages AMASS to discriminate between real human motions and those produced by our temporal pose and shape regression networks. We define a temporal network architecture and show that adversarial training, at the sequence level, produces kinematically plausible motion sequences without in-the-wild ground-truth 3D labels. We perform extensive experimentation to analyze the importance of motion and demonstrate the effectiveness of VIBE on challenging 3D pose estimation datasets, achieving state-of-the-art performance. Code and pretrained models are available at https://github.com/mkocabas/VIBE

Author(s): Muhammed Kocabas and Nikos Athanasiou and Michael J. Black
Book Title: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020)
Pages: 5252--5262
Year: 2020
Month: June
Publisher: IEEE
Project(s):
Bibtex Type: Conference Paper (inproceedings)
Address: Piscataway, NJ
DOI: 10.1109/CVPR42600.2020.00530
Event Name: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020)
Event Place: Seattle, WA, USA
State: Published
Electronic Archiving: grant_archive
ISBN: 978-1-7281-7168-5
Links:

BibTex

@inproceedings{VIBE:CVPR:2020,
  title = {{VIBE}: Video Inference for Human Body Pose and Shape Estimation},
  booktitle = {2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020)},
  abstract = {Human motion is fundamental to understanding behavior. Despite progress on single-image 3D pose and shape estimation, existing video-based state-of-the-art methodsfail to produce accurate and natural motion sequences due to a lack of ground-truth 3D motion data for training. To address this problem, we propose “Video Inference for Body Pose and Shape Estimation” (VIBE), which makes use of an existing large-scale motion capture dataset (AMASS) together with unpaired, in-the-wild, 2D keypoint annotations. Our key novelty is an adversarial learning framework that leverages AMASS to discriminate between real human motions and those produced by our temporal pose and shape
  regression networks. We define a temporal network architecture and show that adversarial training, at the sequence level, produces kinematically plausible motion sequences without in-the-wild ground-truth 3D labels. We perform extensive experimentation to analyze the importance of motion and demonstrate the effectiveness of VIBE on challenging 3D pose estimation datasets, achieving state-of-the-art performance. Code and pretrained models are available at https://github.com/mkocabas/VIBE},
  pages = {5252--5262},
  publisher = {IEEE},
  address = {Piscataway, NJ},
  month = jun,
  year = {2020},
  slug = {vibe-cvpr-2020},
  author = {Kocabas, Muhammed and Athanasiou, Nikos and Black, Michael J.},
  month_numeric = {6}
}