Reinforcement Learning and Control
Model-based Reinforcement Learning and Planning
Object-centric Self-supervised Reinforcement Learning
Self-exploration of Behavior
Causal Reasoning in RL
Equation Learner for Extrapolation and Control
Intrinsically Motivated Hierarchical Learner
Regularity as Intrinsic Reward for Free Play
Curious Exploration via Structured World Models Yields Zero-Shot Object Manipulation
Natural and Robust Walking from Generic Rewards
Goal-conditioned Offline Planning
Offline Diversity Under Imitation Constraints
Learning Diverse Skills for Local Navigation
Learning Agile Skills via Adversarial Imitation of Rough Partial Demonstrations
Combinatorial Optimization as a Layer / Blackbox Differentiation
Object-centric Self-supervised Reinforcement Learning
Symbolic Regression and Equation Learning
Representation Learning
Stepsize adaptation for stochastic optimization
Probabilistic Neural Networks
Learning with 3D rotations: A hitchhiker’s guide to SO(3)
Flowing Puppets
We approach the problem of estimating the pose of charactres in TV shows video sequences.
Our approach is based on optical flow. People are moving entities, and their motion has distinctive characteristics. Often the whole body has a distinctive motion from the background; considering the upper human body, it is also likely that the fastest moving parts are the hands. The articulated structure of the body often generates different motion patterns for each body part. In the recent years the computation of dense optical flow has made large progress in terms of accuracy and computation speed. Based on these observations, we approach the pose estimation problem relying on the dense optical flow as a source of information for better pose estimation.
We precompute the dense optical flow between neighboring frames in the sequences, forward and backward in time. We consider the computed flow as an observation, and exploit it in three ways: a) to estimate hands locations b) to propagate good solutions across frames c) to "link" pose hypotheses in adjiacent frames through the flow to jointly evaluate per-frame image likelihoods.
We represent the upper body with the Deformable Structures model and we exploit its region-based body part representation to estimate how the body moves over time. We call the corresponding moving DS models Flowing Puppets.
Video
Members
Publications