Events & Talks

Perceiving Systems Talk Yandong Wen 05-10-2021 Toward Reconstructing Face from Voice We address a new challenge posed by voice profiling - reconstructing someone’s face from their voice. Specifically, given an audio clip spoken by an unseen person, we aim to reconstruct a face that has as many associations as possible with the speaker in terms of identity. In this talk, I will introduce how we explore and approach the ultimate goal step by step. First, we investigate the audio-visual association by matching voices to faces based on identity, and vice versa. Second, we set up a baseline for reconstructing 2D face images from a voice recording and show reasonable reconstructi... Timo Bolkart

Perceiving Systems Talk Yuxiang Zhang and Yang Zheng 28-09-2021 DeepMultiCap & Lightweight Multi-person Total Motion Capture Using Sparse Multi-view Cameras We propose DeepMultiCap, a novel method for multi-person performance capture using sparse multi-view cameras. Our method can capture time varying surface details without the need of using pre-scanned template models. To tackle the serious occlusion challenge for close interacting scenes, we combine a recently proposed pixel-aligned implicit function with a parametric model for robust reconstruction of the invisible surface areas. An effective attention-aware module is designed to obtain the fine-grained geometry details from multi-view images, where high-fidelity results can be generated. I... Chun-Hao Paul Huang

Perceiving Systems Talk Meng-Yu Jennifer Kuo 27-09-2021 Refraction and Absorption for Underwater Shape Recovery In this talk the speaker will present her work on the recovery of rigid and deformable 3D shape from underwater images. Silvia Zuffi

Perceiving Systems Talk Siwei Zhang 22-09-2021 Learning motion priors for 4D human body capture in 3D scenes It is challenging to recover realistic human-scene interactions and high-quality human motions while dealing with occlusions and partial views with a monocular RGB(D) camera. We address this problem by learning motion smoothness and infilling priors from the large scale mocap dataset AMASS, to reduce the jitters, and handle contacts and occlusions, respectively. Furthermore, we combine them into a multi-stage optimization pipeline for the high quality 4D human capture in complex 3D scenes. Chun-Hao Paul Huang

Perceiving Systems Talk Jiefeng Li 06-09-2021 From skeleton to body: Keypoint Estimation is Helpful for Human Body Reconstruction My works mainly lie in inferring human structures from RGB inputs, which starts from 2D keypoint estimation, towards more complex tasks like 3D skeleton inference and SMPL-based human pose & shape estimation. Along this road, we find that high-level tasks, like human body estimation, can benefit a lot from low-level inferred structures, like 3D skeletons, and vice versa. Furthermore, in our latest work, "Human Pose Regression with Residual Log-likelihood Estimation", we unified all the above HPS tasks in a direct regression paradigm, replacing generally accepted heatmap without loss of accu... Yuliang Xiu

Perceiving Systems Talk Davis Rempe 27-07-2021 Modeling 3D Human Motion for Improved Pose Estimation Though substantial progress has been made in estimating 3D human poses from dynamic observations, recent methods still struggle to recover physically-plausible motions, and the presence of noise and occlusions remains challenging. In this talk, I'll introduce two methods that tackle these issues by leveraging models of 3D human motion - one physics-based and one learned. In the first approach, an initial 3D motion is refined using a physics-based trajectory optimization that leverages automatically-detected foot contacts from RGB video. In the second, a learned generative model is used as a... Muhammed Kocabas

Perceiving Systems Talk Hao Li 26-07-2021 AI SYNTHESIS: FROM AVATARS TO 3D SCENES In this talk I will motivate how digital humans will impact the future of communication, human-machine interaction, and content creation. I will present our latest 3D avatar digitization technology from Pinscreen from a single photo, and give a live demonstration. I will also showcase how we use hybrid CG and neural rendering solutions for real-time applications used in next generation virtual assistant and virtual production pipelines. I will then present a real-time teleportation system that only uses a single webcam as input, and our latest efforts at UC Berkeley in real-time AI synthesi... Yao Feng

Perceiving Systems Talk Maria Kolos 14-07-2021 TRANSPR: Transparency Ray-Accumulating Neural 3D Scene Point Renderer We propose and evaluate a neural point-based graphics method that can model semi-transparent scene parts. Similarly to its predecessor pipeline, ours uses point clouds to model proxy geometry, and augments each point with a neural descriptor. Additionally, a learnable transparency value is introduced in our approach for each point. Our neural rendering procedure consists of two steps. Firstly, the point cloud is rasterized using ray grouping into a multi-channel image. This is followed by the neural rendering step that "translates" the rasterized image into an RGB output using a learnable ... Qianli Ma

Perceiving Systems Talk Nataniel Ruiz 14-06-2021 Using Generative Models for Faces to Test Neural Networks Most machine learning models are validated on fixed datasets. This can give an incomplete picture of the capabilities and weaknesses of the model. Such weaknesses can be revealed at test time in the real world with dire consequences. In order to alleviate this issue, simulators can be controlled in a fine-grained manner using interpretable parameters to explore the semantic image manifold and discover such weaknesses before deploying a model. Also, in recent years there have been important advances in generative models for computer vision resulting in realistic face generation and manipulat... Timo Bolkart

Perceiving Systems Talk Peizhuo Li 10-06-2021 Learning Skeletal Articulations with Neural Blend Shapes Animating a newly designed character using motion capture (mocap) data is a long standing problem in computer animation. A key consideration is the skeletal structure that should correspond to the available mocap data, and the shape deformation in the joint regions, which often requires a tailored, pose-specific refinement. In this work, we develop a neural technique for articulating 3D characters using enveloping with a pre-defined skeletal structure which produces high quality pose dependent deformations. Our framework learns to rig and skin characters with the same articulation structure... Hongwei Yi

Perceiving Systems Talk Marc Habermann 01-06-2021 Real-time Deep Dynamic Characters Animatable and photo-realistic virtual 3D characters are of enormous importance nowadays. However, generating realistic characters still requires manual intervention, expensive equipment, and the resulting characters are either difficult to control or not realistic. Therefore, the goal of the work, that is presented within the talk, is to learn digital characters which are both realistic and easy to control and can be learned directly from a multi-view video. To this end, I will introduce a deep videorealistic 3D human character model displaying highly realistic shape, motion, and dynamic a... Yinghao Huang Chun-Hao Paul Huang

Perceiving Systems Talk Gyeongsik Moon 12-04-2021 Expressive Whole-Body 3D Multi-Person Pose and Shape Estimation from a Single Image Human is the most centric and interesting object in our life: many human-centric techniques and studies have been proposed from both industry and academia, such as virtual try-on, 3D personal avatar, and marker-less motion capture in the movie/game industry, including AR/VR. Recovery of accurate 3D geometry of humans (i.e., 3D human pose and shape) is a key component of the human-centric techniques and studies. In particular, the 3D pose and shape of multiple persons can deliver relative 3D location between persons. Also, the 3D pose and shape of the whole body, which includes hands and fac... Chun-Hao Paul Huang

Perceiving Systems Talk Angjoo Kanazawa 08-04-2021 Pushing the Boundaries of Novel View Synthesis 2020 was a turbulent year, but for 3D learning it was a fruitful one with lots of exciting new tools and ideas. In particular, there have been many exciting developments in the area of coordinate based neural networks and novel view synthesis. In this talk I will discuss our recent work on single image view synthesis with pixelNeRF, which aims to predict a Neural Radiance Field (NeRF) from a single image. I will discuss how NeRF representation allows models like pixel-aligned implicit functions (PiFu) to be trained without explicit 3D supervision and the importance of other key design fact... Qianli Ma

Perceiving Systems Talk Meng Zhang 01-04-2021 Hair & garment synthesis using deep learning method For both AR and VR applications, there is a strong motivation to generate virtual avatars with realistic hairs and garments that are the two most significant elements to personify any character. However, due to the complex structures and ever-changing fashion styles, modeling hairs and garments still remain tedious and expensive as they require considerable professional effort. My research interest focuses on deep learning methods in 3D modeling, rendering, and animation, especially to synthesis high-quality hairs and garments with plausible details. In this talk, I will present the progres... Jinlong Yang

Perceiving Systems Talk Leonidas Guibas 22-02-2021 Joint Learning Over Visual and Geometric Data Many challenges remain in applying machine learning to domains where obtaining massive annotated data is difficult. We discuss approaches that aim to reduce supervision load for learning algorithms in the visual and geometric domains by leveraging correlations among data as well as among learning tasks -- what we call joint learning. The basic notion is that inference problems do not occur in isolation but rather in a "social context" that can be exploited to provide self-supervision by enforcing consistency, thus improving performance and increasing sample efficiency. An example is voting ... Qianli Ma

Perceiving Systems Talk Ruilong Li 10-02-2021 AI Choreographer: Learn to dance with AIST++ In this work, we present a transformer-based learning framework for 3D dance generation conditioned on music. We carefully design our network architecture and empirically study the keys for obtaining qualitatively pleasing results. In addition, we propose a new dataset of paired 3D motion and music called AIST++, which contains 1.1M frames of 3D dance motion in 1408 sequences, covering 10 genres of dance choreographies and accompanied with multi-view camera parameters. To our knowledge it is the largest dataset of this kind. Yuliang Xiu

Perceiving Systems Talk Shruti Agarwal 28-01-2021 Creating, Weaponizing, and Detecting Deep Fakes The past few years have seen a startling and troubling rise in the fake-news phenomena in which everyone from individuals to nation-sponsored entities can produce and distribute misinformation. The implications of fake news range from a misinformed public to an existential threat to democracy, and horrific violence. At the same time, recent and rapid advances in machine learning are making it easier than ever to create sophisticated and compelling fake images, videos, and audio recordings, making the fake-news phenomena even more powerful and dangerous. These AI-synthesized media (so-called... Jinlong Yang

Perceiving Systems Talk Yixin Chen 27-01-2021 Towards a more holistic understanding of scene, object, and human Humans, even young infants, are adept at perceiving and understanding complex indoor scenes. Such an incredible vision system relies on not only the data-driven pattern recognition but also roots from the visual reasoning system, known as the core knowledge, that facilitates the 3D holistic scene understanding tasks. This talk discusses how to employ physical common sense and human-object interaction to bridge scene and human understanding and how the part-level 3D affordance perception may lead to a more fine-grained human-object interaction modeling. Future directions may be extended to d... Dimitris Tzionas

Perceiving Systems Talk Zorah Lähner 21-01-2021 Non-Rigid Shape Correspondence through Deformation Solving for 3D correspondences beyond isometries has made tremendous progress in recent years, much of it due to (deep) learning. However, not all applications provide the necessary training data. This talk will focus on how far we can take the results without learning. I will present a line of work that poses the non-rigid shape registration problem in terms of physical and non-physical deformation energies. Our work aims to combine extrinsic and intrinsic measures to overcome typical shortcomings of both. We use Functional Maps and Markov Chain Monte Carlo initialization to handle all kin... Jinlong Yang

Perceiving Systems Talk Raquel Urtasun 14-12-2020 A Future with Self-Driving Vehicles We are on the verge of a new era in which robotics and artificial intelligence will play an important role in our daily lives. Self-driving vehicles have the potential to redefine transportation as we understand it today. Our roads will become safer and less congested, while parking spots will be repurposed as leisure zones and parks. However, many technological challenges remain as we pursue this future. In this talk I will showcase the latest advancements made by Uber Advanced Technologies Group’s in the quest towards self-driving vehicles at scale. Qianli Ma

Perceiving Systems Talk Daniel Haun 05-10-2020 The phenotyping revolution One of the most striking characteristics of human behavior in contrast to all other animal is that we show extraordinary variability across populations. Human cultural diversity is a biological oddity. More specifically, we propose that what makes humans unique is the nature of the individual ontogenetic process, that results in this unparalleled cultural diversity. Hence, our central question is: How is human ontogeny adapted to cultural diversity and how does it contribute to it? This question is critical, because cultural diversity does not only entail our predominant mode of adaptation ... Timo Bolkart

Perceiving Systems Talk Noah Snavely 02-10-2020 Reconstructing the Plenoptic Function Imagine a futuristic version of Google Street View that could dial up any possible place in the world, at any possible time. Effectively, such a service would be a recording of the plenoptic function—the hypothetical function described by Adelson and Bergen that captures all light rays passing through space at all times. While the plenoptic function is completely impractical to capture in its totality, every photo ever taken represents a sample of this function. I will present recent methods we've developed to reconstruct the plenoptic function from sparse space-time samples of photos—inclu...

Perceiving Systems Talk Daniel Holden 10-08-2020 Functions, Machine Learning, and Game Development Game Development requires a vast array of tools, techniques, and expertise, ranging from game design, artistic content creation, to data management and low level engine programming. Yet all of these domains have one kind of task in common - the transformation of one kind of data into another. Meanwhile, advances in Machine Learning have resulted in a fundamental change in how we think about these kinds of data transformations - allowing for accurate and scalable function approximation, and the ability to train such approximations on virtually unlimited amounts of data. In this talk I will p... Abhinanda Ranjit Punnakkal

Perceiving Systems Talk Vittorio Ferrari 07-08-2020 Our Recent Research on 3D Deep Learning I will present three recent projects within the 3D Deep Learning research line from my team at Google Research: (1) a deep network for reconstructing the 3D shape of multiple objects appearing in a single RGB image (ECCV'20). (2) a new conditioning scheme for normalizing flow models. It enables several applications such as reconstructing an object's 3D point cloud from an image, or the converse problem of rendering an image given a 3D point cloud, both within the same modeling framework (CVPR'20); (3) a neural rendering framework that maps a voxelized object into a high quality image. It re... Yinghao Huang Arjun Chandrasekaran

Perceiving Systems Talk Antonio Torralba 28-07-2020 Learning from vision, touch and audition Babies learn with very little supervision, and, even when supervision is present, it comes in the form of an unknown spoken language that also needs to be learned. How can kids make sense of the world? In this work, I will show that an agent that has access to multimodal data (like vision, audition or touch) can use the correlation between images and sounds to discover objects in the world without supervision. I will show that ambient sounds can be used as a supervisory signal for learning to see and vice versa (the sound of crashing waves, the roar of fast-moving cars – sound conveys impor... Arjun Chandrasekaran

Perceiving Systems Talk Angela Dai 16-07-2020 Towards Commodity 3D Scanning for Content Creation In recent years, commodity 3D sensors have become widely available, spawning significant interest in both offline and real-time 3D reconstruction. While state-of-the-art reconstruction results from commodity RGB-D sensors are visually appealing, they are far from usable in practical computer graphics applications since they do not match the high quality of artist-modeled 3D graphics content. One of the biggest challenges in this context is that obtained 3D scans suffer from occlusions, thus resulting in incomplete 3D models. In this talk, I will present a data-driven approach towards genera... Yinghao Huang

Perceiving Systems Talk William T. Freeman 13-07-2020 Learning from videos played forwards, backwards, fast, and slow How can we tell that a video is playing backwards? People's motions look wrong when the video is played backwards--can we develop an algorithm to distinguish forward from backward video? Similarly, can we tell if a video is sped-up? We have developed algorithms to distinguish forwards from backwards video, and fast from slow. Training algorithms for these tasks provides a self-supervised task that facilitates human activity recognition. We'll show these results, and applications of these unsupervised video learning tasks, including a method to change the timing of people in videos. Yinghao Huang

Perceiving Systems Talk Dushyant Mehta 02-07-2020 Real-time Multi-person 3D Motion Capture with a Single RGB Camera In our recent work, XNect, we propose a real-time solution for the challenging task of multi-person 3D human pose estimation from a single RGB camera. To achieve real-time performance without compromising on accuracy, our approach relies on a new efficient Convolutional Neural Network architecture, and a multi-staged pose formulation. The CNN architecture is approx. 1.3x faster than ResNet-50, while achieving the same accuracy on various tasks, and the benefits extend beyond inference speed to a much smaller training memory footprint and a much higher training throughput. The proposed pose ... Yinghao Huang

Perceiving Systems Talk Srinath Sridhar 10-06-2020 Canonicalization for 3D Perception In this talk, I will introduce the notion of 'canonicalization' and how it can be used to solve 3D computer vision tasks. I will describe Normalized Object Coordinate Space (NOCS), a 3D canonical container that we have developed for 3D estimation, aggregation, and synthesis tasks. I will demonstrate how NOCS allows us to address previously difficult tasks like category-level 6DoF object pose estimation, and correspondence-free multiview 3D shape aggregation. Finally, I will discuss future directions including opportunities to extend NOCS for tasks like articulated and non-rigid shape and po... Timo Bolkart

Perceiving Systems Talk Will Smith 15-05-2020 Deep inverse rendering in the wild In this talk I will consider the problem of scene-level inverse rendering to recover shape, reflectance and lighting from a single, uncontrolled, outdoor image. This task is highly ill-posed, but we show that multiview self-supervision, a natural lighting prior and implicit lighting estimation allow an image-to-image CNN to solve the task, seemingly learning some general principles of shape-from-shading along the way. Adding a neural renderer and sky generator GAN, our approach allows us to synthesise photorealistic relit images under widely varying illumination. I will finish by briefly de... Timo Bolkart

Perceiving Systems Talk Aayush Bansal 14-05-2020 Computational Studio: A computational machinery to enhance social communication Licklider and Taylor (1968) envisioned computational machinery that could enable better communication between humans than face-to-face interaction. In the last fifty years, we have used computing to develop various means of communication, such as mail, messaging, phone calls, video conversation, and virtual reality. These are, however, a proxy of face-to-face communication that aims at encoding words, expressions, emotions, and body language at the source and decoding them reliably at the destination. The true revolution of personal computing has not begun yet because we have not been able ... Arjun Chandrasekaran Chun-Hao Paul Huang

Perceiving Systems Talk Chunyu Wang 16-04-2020 Handling Occlusion by Cross View Fusion for 3D Human Pose Estimation Accurate 3D human pose estimation has been a longstanding goal in computer vision. However, till now, it has only gained limited success in easy scenarios such as studios which have little occlusion. In this talk, I will present our two works aiming to address the occlusion problem in realistic scenarios. In the first work, we present an approach to recover absolute 3D human pose of single person from multi-view images by incorporating multi-view geometric priors in our model. It consists of two separate steps: (1) estimating the 2D poses in multi-view images and (2) recovering the 3D po... Chun-Hao Paul Huang

Perceiving Systems Talk Victoria Fernández Abrevaya 26-03-2020 Learning to Model 3D Human Face Geometry In this talk I will present an overview of our recent works that learn deep geometric models for the 3D face from large datasets of scans. Priors for the 3D face are crucial for many applications: to constrain ill posed problems such as 3D reconstruction from monocular input, for efficient generation and animation of 3D virtual avatars, or even in medical domains such as recognition of craniofacial disorders. Generative models of the face have been widely used for this task, as well as deep learning approaches that have recently emerged as a robust alternative. Barring a few exceptions, mos... Jinlong Yang

Perceiving Systems Talk Prof. Syn Schmitt 06-02-2020 Active motion generation using musculo-skeletal models Biological motion is fascinating in almost every aspect you look upon it. Especially locomotion plays a crucial part in the evolution of life. Structures, like the bones connected by joints, soft and connective tissues and contracting proteins in a muscle-tendon unit enable and prescribe the respective species' specific locomotion pattern. Most importantly, biological motion is autonomously learned, it is untethered as there is no external energy supply and typical for vertebrates, it's muscle-driven. This talk is focused on human motion. Digital models and biologically inspired ... Ahmed Osman

Perceiving Systems Talk Sebastian Starke 28-11-2019 Neural State Machine for Character-Scene Interactions In this talk, I will present about the most recent advances in data-driven character animation and control using neural networks. Creating key-framed animations by hand is typically very time-consuming and requires a lot of artistic expertise and training. Recent work applying deep learning for character animation was firstly able to compete or even outperform the quality that could be achieved by professional animators for biped locomotion, and thus caused a lot excitement in both academia and industry. Shortly after, following research also demonstrated its applicability to quadruped l... Joachim Tesch Ahmed Osman

Perceiving Systems Talk Simone Behrens 21-11-2019 Body models for research on eating behavior, physical activity and body representation The body is one of the most relevant aspects of our self, and we shape it through our eating behavior and physical acitivity. As a psychologist and neuroscientist, I seek to disentangle mutual interactions between how we represent our own body, what we eat and how much we exercise. In the talk, I will give a scoping overview of this approach and present the studies I am conducting as a guest scientist at PS. Ahmed Osman

Perceiving Systems Talk Dr. Tanya Berger-Wolf 23-10-2019 Computational Ecology and AI for Conservation Computation has fundamentally changed the way we study nature. New data collection technology, such as GPS, high definition cameras, UAVs, genotyping, and crowdsourcing, are generating data about wild populations that are orders of magnitude richer than any previously collected. Unfortunately, in this domain as in many others, our ability to analyze data lags substantially behind our ability to collect it. In this talk I will show how computational approaches can be part of every stage of the scientific process of understanding animal sociality, from intelligent data collection (crowdsourci... Aamir Ahmad

Perceiving Systems Talk Fabio Ferreira 27-09-2019 Learning Visual Dynamics Models of Rigid Objects Using Relational Inductive Biases Endowing robots with human-like physical reasoning abilities remains challenging. We argue that existing methods often disregard spatio-temporal relations and by using Graph Neural Networks (GNNs) that incorporate a relational inductive bias, we can shift the learning process towards exploiting relations. In this work, we learn action-conditional forward dynamics models of a simulated manipulation task from visual observations involving cluttered and irregularly shaped objects. We investigate two GNN approaches and empirically assess their capability to generalize to scenarios with novel an... Siyu Tang

Perceiving Systems Talk Eldar Insafutdinov 26-09-2019 Inferring the models of rigid and articulated objects from images: from 2D keypoints to 3D shape and appearance In the first part of the talk, I am going to present our work on human pose estimation in the Wild, capturing unconstrained images and videos containing an a priori unknown number of people, often occluded and exhibiting a wide range of articulations and appearances. Unlike conventional top-down approaches that first detect humans with the off-the-shelf object detector and then estimate poses independently per bounding box, our formulation performs joint detection and pose estimation. In the first stage we indiscriminately localise body parts of every person in the image with the state-of-t... David Hoffmann Siyu Tang

Perceiving Systems Talk Taras Kucherenko 19-09-2019 How to make your agent gesture in a natural way? Conversational agents in the form of virtual agents or social robots are rapidly becoming wide-spread. Humans use non-verbal behaviors to signal their intent, emotions and attitudes in human-human interactions. Conversational agents therefore need this ability as well in order to make an interaction pleasant and efficient. An important part of non-verbal communication is gesticulation: gestures communicate a large share of non-verbal content. Previous systems for gesture production were typically rule-based and could not represent the range of human gestures. Recently the gesture generation... Mohamed Hassan Soubhik Sanyal

Perceiving Systems Talk Björn Browatzki 22-08-2019 Robust Discrimination and Generation of Faces using Compact, Disentangled Embeddings Current solutions to discriminative and generative tasks in computer vision exist separately and often lack interpretability and explainability. Using faces as our application domain, here we present an architecture that is based around two core ideas that address these issues: first, our framework learns an unsupervised, low-dimensional embedding of faces using an adversarial autoencoder that is able to synthesize high-quality face images. Second, a supervised disentanglement splits the low-dimensional embedding vector into four sub-vectors, each of which contains separated information abo... Timo Bolkart

Perceiving Systems Talk Yoshihiro Kanamori 30-07-2019 Relighting Humans: Occlusion-Aware Inverse Rendering for Full-Body Human Images Relighting of human images has various applications in image synthesis. For relighting, we must infer albedo, shape, and illumination from a human portrait. Previous techniques rely on human faces for this inference, based on spherical harmonics (SH) lighting. However, because they often ignore light occlusion, inferred shapes are biased and relit images are unnaturally bright particularly at hollowed regions such as armpits, crotches, or garment wrinkles. This paper introduces the first attempt to infer light occlusion in the SH formulation directly. Based on supervised learning using conv... Senya Polikovsky Jinlong Yang

Perceiving Systems Talk Chengde Wan 23-07-2019 Self-supervised 3D hand pose estimation Deep learning has significantly advanced state-of-the-art for 3D hand pose estimation, of which accuracy can be improved with increased amounts of labelled data. However, acquiring 3D hand pose labels can be extremely difficult. In this talk, I will present our recent two works on leveraging self-supervised learning techniques for hand pose estimation from depth map. In both works, we incorporate differentiable renderer to the network and formulate training loss as model fitting error to update network parameters. In first part of the talk, I will present our earlier work which approximates... Dimitris Tzionas

Perceiving Systems Talk Shunsuke Saito 10-07-2019 Robust and High-Fidelity Human Digitization from Minimal Inputs Realistic digital avatars are increasingly important in digital media with potential to revolutionize 3D face-to-face communication and social interactions through compelling digital embodiment of ourselves. My goal is to efficiently create high-fidelity 3D avatars from a single image input, captured in an unconstrained environment. These avatars must be close in quality to those created by professional capture systems, yet require minimal computation and no special expertise from the user. These requirements pose several significant technical challenges. A single photograph provides only p... Timo Bolkart

Perceiving Systems Talk Dr Antonia Tzemanaki 05-06-2019 Anthropomorphism in Surgical Robotics and Wearable Technologies Over the past century, abdominal surgery has seen a rapid transition from open procedures to less invasive methods such as laparoscopy and robot-assisted minimally invasive surgery (R-A MIS), as they involve reduced blood loss, postoperative morbidity and length of hospital stay. Furthermore, R-A MIS has offered refined accuracy and more ergonomic instruments for surgeons, further minimising trauma to the patient. However, training surgeons in MIS procedures is becoming increasingly long and arduous, while commercially available robotic systems adopt a design similar to conventional laparos... Dimitris Tzionas

Perceiving Systems Talk Jinlong Yang 03-05-2019 Modeling shape space of clothed human using a learning approach In the past few years, significant progress has been made on shape modeling of human body, face, and hands. Yet clothing shape is currently not well presented. Modeling clothing using physics-based simulation can sometimes involve tedious manual work and heavy computation. Therefore, a data-driven learning approach has emerged in the community. In this talk, I will present a stream of work that targeted to learn the shape of clothed human from captured data. It involves 3D body estimation, clothing surface registration and clothing deformation modeling. I will conclude this talk by outlini... Timo Bolkart

Perceiving Systems Talk Marilyn Keller 26-04-2019 Using RGB-D cameras for scene streaming and navigation in virtual reality Since the release of the Kinect, RGB-D cameras have been used in several consumer devices, including smartphones. In this talk, I will present two challenging uses of this technology. With multiple RGB-D cameras, it is possible to reconstruct a 3D scene and visualize it from any point of view. In the first part of the talk, I will show how such a scene can be streamed and rendered as a point cloud in a compelling way and its appearance improved by the use of external cinema cameras. In the second part of the talk, I will present my work on how an RGB-D camera can be used for enabling real-w... Sergi Pujades

Perceiving Systems IS Colloquium Peter Dayan 21-12-2018 Mind Games Much existing work in reinforcement learning involves environments that are either intentionally neutral, lacking a role for cooperation and competition, or intentionally simple, when agents need imagine nothing more than that they are playing versions of themselves. Richer game theoretic notions become important as these constraints are relaxed. For humans, this encompasses issues that concern utility, such as envy and guilt, and that concern inference, such as recursive modeling of other players, I will discuss studies treating a paradigmatic game of trust as an interactive partia...

Perceiving Systems Talk Stefanos Zafeiriou 17-12-2018 Generating Faces & Heads: Texture, Shape and Beyond. The past few years with the advent of Deep Convolutional Neural Networks (DCNNs), as well as the availability of visual data it was shown that it is possible to produce excellent results in very challenging tasks, such as visual object recognition, detection, tracking etc. Nevertheless, in certain tasks such as fine-grain object recognition (e.g., face recognition) it is very difficult to collect the amount of data that are needed. In this talk, I will show how, using DCNNs, we can generate highly realistic faces and heads and use them for training algorithms such as face and facial express... Dimitris Tzionas

Perceiving Systems Talk Yanxi Liu 12-12-2018 Learning Dynamics from Kinematics: Estimating Foot Pressure from Video Human pose stability analysis is the key to understanding locomotion and control of body equilibrium, with numerous applications in the fields of Kinesiology, Medicine and Robotics. We propose and validate a novel approach to learn dynamics from kinematics of a human body to aid stability analysis. More specifically, we propose an end-to-end deep learning architecture to regress foot pressure from a human pose derived from video. We have collected and utilized a set of long (5min +) choreographed Taiji (Tai Chi) sequences of multiple subjects with synchronized motion capture, foot pressure ... Nadine Rueegg