Perceiving Systems – Max Planck Institute for Intelligent Systems

Institute Homepage Sign In

Thumb xl 1 20240715 perc syst 129

Thumb xl 1 20240715 perc syst 113

Thumb xl 3 adventscalend ar 9504x6336

Thumb xl 4 20240715 perc syst 049

Thumb xl 5 20240715 perc syst 013

Thumb xl 6 20240715 perc syst 081

Thumb xl 8 20240715 perc syst 126

Thumb xl 9 20240715 perc syst 087

Thumb xl 10 20240715 perc syst 009

Thumb xl 11 20240715 perc syst 080

Thumb md 1 20240715 perc syst 129

Thumb md 1 20240715 perc syst 113

Thumb md 3 adventscalend ar 9504x6336

Thumb md 4 20240715 perc syst 049

Thumb md 5 20240715 perc syst 013

Thumb md 6 20240715 perc syst 081

Thumb md 8 20240715 perc syst 126

Thumb md 9 20240715 perc syst 087

Thumb md 10 20240715 perc syst 009

Thumb md 11 20240715 perc syst 080

Our research uses Computer Vision to learn digital humans that can perceive, learn, and act in virtual 3D worlds. This involves capturing the shape, appearance, and motion of real people as well as their interactions with each other and the 3D scene using monocular video. We leverage this to learn generative models of people and their behavior and evaluate these models by synthesizing realistic looking humans behaving in virtual worlds.

This work combines Computer Vision, Machine Learning, and Computer Graphics.

Director

Michael Black

Michael Black

Director

Admin Team

Melanie Feldhofer

Melanie Feldhofer

Department Manager

Nicole Overbaugh

Nicole Overbaugh

Administrative Assistant

+49 7071 601 1800

ps-admin@tuebingen.mpg.de

Department Highlights

Publication Perceiving Systems 2025-03-01

CameraHMR: Aligning People with Perspective

{CameraHMR}: Aligning People with Perspective

Publication Perceiving Systems 2025-01-01

Democratizing 3D Human Digitization

Democratizing 3D Human Digitization

Publication Perceiving Systems 2024-12-01

PuzzleAvatar: Assembling 3D Avatars from Personal Albums

{PuzzleAvatar}: Assembling {3D} Avatars from Personal Albums

Talk Perceiving Systems 2024-11-28

How to predict the inside from the outside? Segment, register, model and infer!

How to predict the inside from the outside? Segment, register, model and infer!

Publication Haptic Intelligence Perceiving Systems 2024-11-01

Wrist-to-Wrist Bioimpedance Can Reliably Detect Discrete Self-Touch

Wrist-to-Wrist Bioimpedance Can Reliably Detect Discrete Self-Touch

Talk Perceiving Systems 2024-10-14

Diffusion Models for Human Motion Synthesis

Diffusion Models for Human Motion Synthesis

Talk Perceiving Systems 2024-10-10

Reconstruction and Animation of Realistic Head Avatars

Reconstruction and Animation of Realistic Head Avatars

Talk Perceiving Systems 2024-09-26

Collaborative Control for Geometry-Conditioned PBR Image Generation

Collaborative Control for Geometry-Conditioned PBR Image Generation

Talk Perceiving Systems 2024-09-26

Geometry Image Diffusion: Fast and Data-Efficient Text-to-3D with Image-Based Surface Representation

Geometry Image Diffusion: Fast and Data-Efficient Text-to-3D with Image-Based Surface Representation

Talk Perceiving Systems 2024-09-12

Generalizable Object-aware Human Motion Synthesis

Generalizable Object-aware Human Motion Synthesis

Publication Perceiving Systems 2024-09-01

AWOL: Analysis WithOut synthesis using Language

{AWOL: Analysis WithOut synthesis using Language}

Publication Perceiving Systems 2024-09-01

GraspXL: Generating Grasping Motions for Diverse Objects at Scale

{GraspXL}: Generating Grasping Motions for Diverse Objects at Scale

Publication Perceiving Systems 2024-09-01

Explorative Inbetweening of Time and Space

Explorative Inbetweening of Time and Space

Talk Perceiving Systems 2024-08-22

Real Virtual Humans

Real Virtual Humans

Publication Empirical Inference Perceiving Systems 2024-08-15

Can Large Language Models Understand Symbolic Graphics Programs?

Publication Perceiving Systems 2024-08-01

Re-Thinking Inverse Graphics with Large Language Models

Re-Thinking Inverse Graphics with Large Language Models

Talk Perceiving Systems 2024-07-25

4D Dynamic Scene Reconstruction, Editing, and Generation.

4D Dynamic Scene Reconstruction, Editing, and Generation.

Talk Perceiving Systems 2024-07-23

Multimodal Social Signal Processing for Human-Robot Interaction

Multimodal Social Signal Processing for Human-Robot Interaction

Talk Perceiving Systems 2024-07-18

Integrating AI Agents into Human Lives via a Simulation Approach

Integrating AI Agents into Human Lives via a Simulation Approach

Talk Perceiving Systems 2024-07-18

Recreating Real Garments in Virtual Space with Gaussian Splatting and GNNs

Recreating Real Garments in Virtual Space with Gaussian Splatting and GNNs

Talk Perceiving Systems 2024-07-08

Creating High-End Visuals with Real-Time Technology

Creating High-End Visuals with Real-Time Technology

Talk Perceiving Systems 2024-07-04

Text-Driven 3D Modeling of Avatars

Text-Driven 3D Modeling of Avatars

Publication Perceiving Systems 2024-07-01

ContourCraft: Learning to Resolve Intersections in Neural Multi-Garment Simulations

{ContourCraft}: Learning to Resolve Intersections in Neural Multi-Garment Simulations

Talk Perceiving Systems 2024-06-10

Towards Human-Centric Foundation Models: Pretraining Datasets and Unified Architectures

Towards Human-Centric Foundation Models: Pretraining Datasets and Unified Architectures

Publication Perceiving Systems 2024-06-01

MonoHair: High-Fidelity Hair Modeling from a Monocular Video

{MonoHair}: High-Fidelity Hair Modeling from a Monocular Video

Publication Perceiving Systems 2024-06-01

SMIRK: 3D Facial Expressions through Analysis-by-Neural-Synthesis

{SMIRK}: {3D} Facial Expressions through Analysis-by-Neural-Synthesis

Publication Perceiving Systems Neural Capture and Synthesis Human-centric Vision & Learning 2024-06-01

Text-Conditioned Generative Model of 3D Strand-based Human Hairstyles

Text-Conditioned Generative Model of 3D Strand-based Human Hairstyles

Publication Perceiving Systems 2024-06-01

Generative Proxemics: A Prior for 3D Social Interaction from Images

Generative Proxemics: A Prior for {3D} Social Interaction from Images

Publication Perceiving Systems 2024-06-01

WANDR: Intention-guided Human Motion Generation

{WANDR}: Intention-guided Human Motion Generation

Publication Perceiving Systems 2024-06-01

HIT: Estimating Internal Human Implicit Tissues from the Body Surface

{HIT}: Estimating Internal Human Implicit Tissues from the Body Surface

Publication Perceiving Systems 2024-06-01

HOLD: Category-agnostic 3D Reconstruction of Interacting Hands and Objects from Video

{HOLD}: Category-agnostic {3D} Reconstruction of Interacting Hands and Objects from Video

Publication Perceiving Systems 2024-06-01

SCULPT: Shape-Conditioned Unpaired Learning of Pose-dependent Clothed and Textured Human Meshes

{SCULPT}: Shape-Conditioned Unpaired Learning of Pose-dependent Clothed and Textured Human Meshes

Publication Perceiving Systems 2024-06-01

AMUSE: Emotional Speech-driven 3D Body Animation via Disentangled Latent Diffusion

{AMUSE}: Emotional Speech-driven {3D} Body Animation via Disentangled Latent Diffusion

Publication Perceiving Systems 2024-06-01

WHAM: Reconstructing World-grounded Humans with Accurate 3D Motion

{WHAM}: Reconstructing World-grounded Humans with Accurate {3D} Motion

Publication Perceiving Systems 2024-06-01

ChatPose: Chatting about 3D Human Pose

{ChatPose}: Chatting about 3D Human Pose

Publication Perceiving Systems 2024-06-01

EMAGE: Towards Unified Holistic Co-Speech Gesture Generation via Expressive Masked Audio Gesture Modeling

{EMAGE}: Towards Unified Holistic Co-Speech Gesture Generation via Expressive Masked Audio Gesture Modeling

Talk Perceiving Systems 2024-05-02

Generative Rendering and Beyond

Generative Rendering and Beyond

Publication Empirical Inference Perceiving Systems 2024-05-01

Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization

Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization

Publication Perceiving Systems 2024-05-01

The Poses for Equine Research Dataset (PFERD)

The Poses for Equine Research Dataset {(PFERD)}

Publication Perceiving Systems Empirical Inference 2024-05-01

Ghost on the Shell: An Expressive Representation of General 3D Shapes

Ghost on the Shell: An Expressive Representation of General {3D} Shapes

Talk Perceiving Systems 2024-04-04

Modeling and Reconstructing Garments with Sewing Patterns

Modeling and Reconstructing Garments with Sewing Patterns

Talk Perceiving Systems 2024-03-13

Geometric Regularizations for 3D Shape Generation

Geometric Regularizations for 3D Shape Generation

Publication Perceiving Systems 2024-03-01

TADA! Text to Animatable Digital Avatars

{TADA!} Text to Animatable Digital Avatars

Publication Perceiving Systems 2024-03-01

TeCH: Text-guided Reconstruction of Lifelike Clothed Humans

{TeCH}: Text-guided Reconstruction of Lifelike Clothed Humans

Publication Perceiving Systems Neural Capture and Synthesis 2024-03-01

TECA: Text-Guided Generation and Editing of Compositional 3D Avatars

{TECA}: Text-Guided Generation and Editing of Compositional {3D} Avatars

Talk Perceiving Systems 2024-01-18

Mining Visual Knowledge from Large Pre-trained Models

Mining Visual Knowledge from Large Pre-trained Models

Publication Perceiving Systems 2024-01-01

InterCap: Joint Markerless 3D Tracking of Humans and Objects in Interaction from Multi-view RGB-D Images

{InterCap}: Joint Markerless {3D} Tracking of Humans and Objects in Interaction from Multi-view {RGB-D} Images

Publication Perceiving Systems 2024-01-01

Human Hair Reconstruction with Strand-Aligned 3D Gaussians

Human Hair Reconstruction with Strand-Aligned {3D} Gaussians

Publication Perceiving Systems 2023-12-01

FLARE: Fast learning of Animatable and Relightable Mesh Avatars

{FLARE}: Fast learning of Animatable and Relightable Mesh Avatars

Publication Perceiving Systems 2023-12-01

From Skin to Skeleton: Towards Biomechanically Accurate 3D Digital Humans

From Skin to Skeleton: Towards Biomechanically Accurate {3D} Digital Humans