Max Planck Institute for Intelligent Systems

Perceiving Systems PS:License 1.0 Capturing Hand-Object Interaction and Reconstruction of Manipulated Objects Webpages for the GCPR 2013, GCPR 2014, ICCV 2015, IJCV 2016, ECCVw 2016 papers. The data contains: (IJCV 2016, GCPR 2014) annotated RGB-D and multicamera-RGB dataset of one or two hands interacting with each other and/or with a rigid or an articulated object, (ICCV 2015) RGB-D dataset of a hand rotating a rigid object for 3d scanning, (GCPR 2013) synthetic dataset of two hands interacting with each other, (ECCVw 2016) RGB-D dataset of an object under manipulation.

Perceiving Systems PS:License 1.0 SMPLify: 3D human pose and shape estimation from a single image Given a single image, extract the 3D SMPL pose and shape parameters. We provide a Python demo code needed to run SMPLify. We also provide results from the ECCV paper for comparison. For all the datasets we used (LSP, HumanEva-I, Human3.6M) we provide the detected joints and our results as SMPL model parameters and as a mesh (vertices and faces). The code package includes an example script showing how to load results. Please see the README in the code package and the FAQ.

Perceiving Systems PS:License 1.0 BodyTalk: Tool that relates 3D body shape to words This website provides a tool to explore 3D body shape and linguistic descriptions of shape. We provide a set of shape sliders and linguistic sliders that can be used to change body shape. This allows you to explore how people think about body shape and how shape and adjectives are correlated.

Perceiving Systems PS:License 1.0 Semantic Optical Flow Data and code necessary to reproduce results from the CVPR 2016 paper on semantic optical flow. Semantic scene segmentation enables different flow models to be used in different regions and then composed using a locally layered approach.

Perceiving Systems PS:License 1.0 Video segmentation via object flow Matlab implementation of the paper Video Segmentation via Object Flow Yi-Hsuan Tsai, Ming-Hsuan Yang and Michael J. Black IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.

Perceiving Systems PS:License 1.0 SMPL: A skinned multi-person linear body model SMPL is like a PDF format for 3D bodies. It is a realistic 3D model of the human body that is based on blend skinning and blend shapes that is learned from thousands of 3D body scans. It is fully portable, works with many existing game engines and is useful for computer vision. This site provides resources to learn about SMPL, including example FBX files with animated SMPL models, and code for using SMPL in Python, Maya and Unity. The Python code shows how to use SMPL in computer vision problems. Maya and Unity scripts help set up the model for animation in these 3D environments. We p...

Perceiving Systems PS:License 1.0 MoSh: Motion and Shape Capture from Sparse Markers The original MoSh dataset is available as well as code for the latest MoSh++ method. Marker-based motion capture (mocap) is widely criticized as producing lifeless animations. MoSh (Motion and Shape capture), automatically extracts detail present in the original mocap maker data. MoSh estimates body shape and pose together using sparse marker data by exploiting a parametric model of the human body. The dataset contains: 1) The original .c3d files with MOCAP marker-data. 2) Estimated 3D shape meshes. 3) 3D scans from a high resolution scanner for comparison. The code is the latest MoSh++ m...

Perceiving Systems PS:License 1.0 Dyna: 4D meshes of dynamic human soft tissue motion To look human, digital full-body avatars need to have soft tissue deformations like those of real people. Current methods for physics simulation of soft tissue lack realism, are computationally expensive, or are hard to tune. Learning soft tissue motion from example, however, has been limited by the lack of dense, high-resolution, training data. We address this using a 4D capture system and a method for accurately registering 3D scans across time to a template mesh. Using over 40,000 scans of ten subjects, we compute how soft tissue motion causes mesh triangles to deform relative to a base ...

Software Workshop The MIT License Livius The tool we developed for generating the videos on MLSS 2015

Perceiving Systems PS:License 1.0 Pose-Conditioned Joint Angle Limits for 3D Human Pose Reconstruction (code and data) This release contains code and data. Most mocap datasets are too small or to constrained to capture the full range of human motions. In particular, they are too small to explore joint angle limits. Here we provide a mocap dataset in which the subjects are gymnasts who are able to explore a wide range of human poses. The dataset allows one to develop pose priors that obey these limits and to model how these joint limits actually vary with pose. We include code to learn joint angle limits and to estimate 3D pose from 2D joint locations.

Perceiving Systems Autonomous Vision PS:License 1.0 KITTI 2015: Stereo, Flow, and Scene Flow Benchmark KITTI is one of the most popular datasets for evaluation of vision algorithms, particuarly in the context of street scenes and autonomous driving. The stereo 2015 / flow 2015 / scene flow 2015 benchmark consists of 200 training scenes and 200 test scenes (4 color images per scene, saved in loss less png format). Compared to the stereo 2012 and flow 2012 benchmarks, it comprises dynamic scenes for which the ground truth has been established in a semi-automatic process.

Perceiving Systems PS:License 1.0 PCA-Flow: Fast, approximate optical flow computation This software package contains two algorithms for the computation of optical flow, as described in Wulff & Black, "Efficient Sparse-to-Dense Optical Flow Estimation using a Learned Basis and Layers" (CVPR 2015). PCA-Flow computes approximate optical flow extremely quickly, by making the assumption that optical flow lies on a low-dimensional subspace. PCA-Layers extends this to a layered model to increase accuracy, especially at boundaries. It is the most accurate layered model on the MPI Sintel dataset.

Perceiving Systems PS:License 1.0 The Stitched Puppet: A Graphical Model of 3D Human Shape and Pose The Stitched Puppet (SP) is a realistic part-based 3D body model of the human body. It offers the best features of part-based body models used in Computer Vision and statistical body models used in Computer Graphics. The release includes data and code to fit the SP model to 3D scans.

Software Workshop The 3-Clause BSD License Code Doc A Django application for hosting and accessing documentation and distributions generated from source code.

Perceiving Systems PS:License 1.0 OpenDR: An open differentiable renderer The OpenDR, is the first open source differentiable renderer. It provides a simple Python interface for defining an objective function with a forward generative process and then automatically differentiating and optimizing this. OpenDR allows for quick design and testing of generative models in computer vision. The code provides examples. OpenDR has been widely use.

Perceiving Systems PS:License 1.0 FAUST dataset: High-resolution 3D scans with ground truth correspondence FAUST contains 300 real, high-resolution human scans of 10 different subjects in 30 different poses, with automatically computed ground-truth correspondences. We provide a training set with scans and ground truth correspondence. We also provide a separate test set of scans with an evaluation website that compares results of mesh correspondence.

Perceiving Systems Software Workshop The 3-Clause BSD License Robust and scalable PCA using Grassmann averages The Grassmann Averages PCA is a method for extracting the principal components from a sets of vectors, with the nice following properties: 1) it is of linear complexity wrt. the dimension of the vectors and the size of the data, which makes the method highly scalable, 2) It is more robust to outliers than PCA in the sense that it minimizes an L1 norm instead of the L2 norm of the standard PCA. It comes with two variants: 1) the standard computation, that coincides with the PCA for normally distributed data, also referred to as the GA, 2) a trimmed variant, that is more robust to outliers...

Perceiving Systems PS:License 1.0 Secrets of Optical Flow: Code for various methods Matlab code for robust optical flow -- Classic++ and Classic-NL -- as described in the IJCV paper "A Quantitative Analysis of Current Practices in Optical Flow Estimation and the Principles behind Them". This code is widely used as a baseline and starting point for "classical" flow methods. Matlab version of the "Black and Anandan" robust flow method: https://deqings.github.io/public_files/ba.zip Matlab version of "Horn and Schunck": https://deqings.github.io/public_files/hs.zip Original implementation from CVPR'2010 paper: https://deqings.github.io/public_files/cvpr10_flow_code.zip

Perceiving Systems PS:License 1.0 Flowing Puppets Code for ICCV'13 paper on "Estimating Human Pose with Flowing Puppets". This addresses the problem of upper-body human pose estimation in uncontrolled monocular video sequences, without manual initialization. The "flowing puppets" model provide integrates image evidence across frames to improve pose inference. We provide the code used for the experiments in the paper. We also provide the "puppet flow" annotation tool.

Perceiving Systems PS:License 1.0 JHMDB: Joint-annotated Human Motion Data Base A fully annotated data set for human actions and human poses. It is based on the HMDB human motion dataset but includes optical flow on the person, the segmentation of the person, joint locations, action labels, and meta data.

Perceiving Systems PS:License 1.0 BMI Visualizer This website helps people understand body mass index through a novel visualization of 3D body shape. Enter height and weight to see a 3D body shape with these properties and see the corresponding BMI. Move a slider to change BMI and see how body shape changes.

Perceiving Systems Autonomous Vision PS:License 1.0 The KITTI Dataset The KITTI dataset is the de-facto standard for developing and testing computer vision algorithms for real-world autonomous driving scenarios and more.

Perceiving Systems PS:License 1.0 Sintel Optical Flow Dataset and Benchmark The MPI Sintel Dataset is one of the most widely used datasets for training and evaluating optical flow algorithms. It is the first synthetic dataset to achieve wide use because of it well represents natural scenes and motions. It is also extremely challenging and current methods have still not fully "solved" the problem of flow estimation for Sintel. Sintel is designed to encourage research on long-range motion, motion blur, multi-frame analysis, non-rigid motion. Algorithms are evaluated on held-out test data and results are displayed for comparison. The dataset contains flow field...

Perceiving Systems PS:License 1.0 Lie Bodies This code supports the core representation needed for Lie Bodies as described in Freifeld and Black, ECCV 2012. Currently this is only a partial version of what is presented in the paper. The code takes pairs of triangles and computes the "Q" matrices and the corresponding (R,A,S) decompositions that are the foundation of Lie Bodies (see paper for details).

Perceiving Systems PS:License 1.0 Body Shape Visualizer This web-based tool lets users enter information about body measurements (height, waist, inseam, etc) and visualize a 3D body shape that corresponds to these measurements.

Perceiving Systems PS:License 1.0 Middlebury Optical Flow Dataset and Benchmark The Middlebury flow dataset has been a de-facto standard in the field since 2007. The dataset introduced several innovations. It is the first dataset to contain real image sequences with independent motions, and ground truth optical flow. Second, it provides realistically rendered synthetic scenes with ground truth flow. It also includes a frame interpolation task using real video sequences. While, by today's standards, the dataset is small and the sequences somewhat simple, it remains a useful tool for evaluating the generality of optical flow methods.

Perceiving Systems PS:License 1.0 HumanEva datasets I and II HumanEva: Synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion. This is the repository for the widely use HumanEva dataset. This was the first dataset to include mutii-camera video capture of people with ground truth 3D human pose. It established the quantitative evaluation of human pose estimation using well-defined metrics in 2D and 3D. The dataset was developed at Brown University and is hosted by MPI.

Perceiving Systems PS:License 1.0 Archival Image sequences This is a collection of images sequences used in the 1990's and early 2000's. It includes the Yosemite sequences, Flower garden, and several others.

Perceiving Systems PS:License 1.0 Robust dense optical flow estimation This is the original C implementation of the "Black and Anandan" optical flow algorithm. For a Matlab version that is written by Deqing Sun and much better see: https://deqings.github.io/public_files/ba.zip

Empirical Inference The 3-Clause BSD License Half-Sibling Regression for High-Contrast Imaging Methods for applying a post-processing scheme based on Half-Sibling Regression (HSR) to High-Contrast Imaging (HCI) data

Empirical Inference The MIT License AutoML Two-Sample Test autotst is a Python package for easy-to-use two-sample testing and distribution shift detection.

The MIT License AutoML Two-Sample Test autotst is a Python package for easy-to-use two-sample testing and distribution shift detection.

Autonomous Motion GNU General Public License version 3 Bayesian Object Tracking Robust and real-time Bayesian articulated object tracking methods, implemented in C++ and CUDA.

Empirical Inference The MIT License Dingo (Deep Inference for Gravitational-wave Observations) Dingo (Deep Inference for Gravitational-wave Observations) is a Python program for analyzing gravitational wave data using neural posterior estimation. It dramatically speeds up inference of astrophysical source parameters from data measured at gravitational-wave observatories. Dingo aims to enable the routine use of the most advanced theoretical models in analysing data, to make rapid predictions for multi-messenger counterparts, and to do so in the context of sensitive detectors with high event rates.

Autonomous Vision OctNet: Learning Deep 3D Representations at High Resolutions We present OctNet, a representation for deep learning with sparse 3D data. In contrast to existing models, our representation enables 3D convolutional networks which are both deep and high resolution. Towards this goal, we exploit the sparsity in the input data to hierarchically partition the space using a set of unbalanced octrees where each leaf node stores a pooled feature representation. This allows to focus memory allocation and computation to the relevant dense regions and enables deeper networks without compromising resolution. We demonstrate the utility of our OctNet representation ...

Empirical Inference The MIT License Omni-Fig: Unleashing Project Configuration and Organization in Python omni-fig is a lightweight package to help you organize your python projects to make everything clear and easy to understand to collaborators and prospective users, while also offering unparalleled features to accelerate development. The proposed general-purpose project structure is well suited for both small and large projects, and is designed to be easily extensible to fit your needs. Most importantly, with the powerful configuration system, you never have to worry about any boilerplate code to parse command line arguments, read config files, or even import the top-level project compone...

Perceiving Systems PS:License 1.0 Robust area-based optical flow estimation This is the original C version of the robust area-based flow code from Michael Black's thesis.

Empirical Inference The 3-Clause BSD License The o80 C++ templated toolbox for robotics o80 (pronounced "oh-eighty") is software for synchronizing and organizing message exchange between (realtime) processes via simple customized Python APIs. Its target domain is robotics and machine learning. Our motivation for developing o80 is to ease the setup of robotics experiments (i.e., integration of various hardware and software) by machine learning scientists. Such setup typically requires time and technical effort, especially when realtime processes are involved. Ideally, scientists should have access to a simple Python API that hides the lower level communication details and simpl...

Social Foundations of Computation The MIT License folktables Datasets derived from US census data

Empirical Inference The MIT License normflows: A PyTorch Package for Normalizing Flows normflows is a PyTorch implementation of discrete normalizing flows. Many popular flow architectures are implemented. The package can be easily installed via pip. The basic usage is described here, and a full documentation is available as well. A more detailed description of this package is given in out accompanying paper.

Research

Departments

Research Groups

People

Contact

Our Institute

Our History

Career

Doctoral Programs

Training

Service Units

Central Scientific Facilities

Workshops

Campus Services

Impact

Cooperation

Partners and Initiatives