Human Pose, Shape and Action
3D Pose from Images
2D Pose from Images
Beyond Motion Capture
Action and Behavior
Body Perception
Body Applications
Pose and Motion Priors
Clothing Models (2011-2015)
Reflectance Filtering
Learning on Manifolds
Markerless Animal Motion Capture
Multi-Camera Capture
2D Pose from Optical Flow
Body Perception
Neural Prosthetics and Decoding
Part-based Body Models
Intrinsic Depth
Lie Bodies
Layers, Time and Segmentation
Understanding Action Recognition (JHMDB)
Intrinsic Video
Intrinsic Images
Action Recognition with Tracking
Neural Control of Grasping
Flowing Puppets
Faces
Deformable Structures
Model-based Anthropometry
Modeling 3D Human Breathing
Optical flow in the LGN
FlowCap
Smooth Loops from Unconstrained Video
PCA Flow
Efficient and Scalable Inference
Motion Blur in Layers
Facade Segmentation
Smooth Metric Learning
Robust PCA
3D Recognition
Object Detection
Human health and the 3D body
].Our 3D body models and tools for estimating them from data are the world's most accurate and are widely used for evaluating body composition and associated health risk factors. Can we leverage body shape to further improve human health? What does the surface of the body tell us about our risk of disease? Can we leverage 3D humans in the treatment of disease? To answer these questions, we develop new technologies and partner with experts in medicine and psychology.
Looking inside: Can we predict the inside of the body from the outside? OSSO [] predicts the bones inside the body from the 3D body surface. This is challenging due to the lack of paired data of 3D bodies and their skeletons. To address this we built a unique dataset by fitting 3D bodies and skeletongs to DXA scans. SKEL [] goes further to estimate the skeleton in motion from the body surface using a novel BioAMASS dataset (see Dataset research field). SKEL also introduces a new body model based on SMPL in which the pose is controlled by a biomechanical skeleton. We also used our technology to model the shape the humerus to enable surgeons to automatically create custom plates for the repair of humerus fractures []. Finally, HIT [] infers an implicit representation for the lean and adipose tissues in the body from the surface by exploiting a new dataset of segmented MRI images.
Body shape and anorexia nervosa: Treatment results of anorexia nervosa (AN) are modest, with fear of weight gain being a strong predictor of treatment outcome and relapse. Continuing a long line of research, we developed a virtual reality (VR) setup for exposure to healthy weight bodies and observed that repeated virtual exposure to healthy weight improved fear of weight gain, suggesting a possible adjunct treatment for AN [].
Sign language (SL) is the primary method of communication for the 70 million Deaf people around the world. Signing avatars could aid learning and improving access to technology and online media. Training such avatars requires capturing 3D data from video at scale. SGNify [] captures fine-grained hand pose, facial expression, and body movement fully automatically from in-the-wild monocular SL videos.