Events & Talks

Perceiving Systems IS Colloquium 05-05-2014 Video Segmentation Compared to static image segmentation, video segmentation is still in its infancy. Various research groups have different tasks in mind when they talk of video segmentation. For some it is motion segmentation, some think of an over-segmentation with thousands of regions per video, and others understand video segmentation as contour tracking. I will go through what I think are reasonable video segmentation subtasks and will touch the issue of benchmarking. I will also discuss the difference between image and video segmentation. Due to the availability of motion and the redundancy of successi... Gerard Pons-Moll

Perceiving Systems Talk 30-04-2014 Large displacement optical flow & flow-based action recognition In the first part of our talk, we present an approach for large displacement optical flow. Optical flow computation is a key component in many computer vision systems designed for tasks such as action detection or activity  recognition. Inspired by the large displacement optical flow of Brox and  Malik, our approach  DeepFlow  combines a novel matching algorithm with a variational approach . Our matching algorithm builds upon a multi-stage architecture interleaving convolutions and max-pooling.  DeepFlow efficiently handles large displacements&nbsp...

Perceiving Systems IS Colloquium 28-04-2014 WaldBoost: Combining Sequential Analysis with Machine Learning for Solving Time-constrained Vision Problems Computer vision problems often involve optimization of two quantities, one of which is time. Such problems can be formulated as time-constrained optimization or performance-constrained search for the fastest algorithm. We show that it is possible to obtain quasi-optimal time-constrained solutions to some vision problems by applying Wald's theory of sequential decision-making. Wald assumes independence of observation, which is rarely true in computer vision. We address the problem by combining Wald's sequential probability ratio test and AdaBoost. The solution, called the WaldBoost... Gerard Pons-Moll

Perceiving Systems Talk 10-04-2014 Scalable Surface-Based Stereo Matching Stereo matching -- establishing correspondences between images taken from nearby viewpoints -- is one of the oldest problems in computer vision.  While impressive progress has been made over the last two decades, most current stereo methods do not scale to the high-resolution images taken by today's cameras since they require searching the full space of all possible disparity hypotheses over all pixels. In this talk I will describe a new scalable stereo method that only evaluates a small portion of the search space.  The method first generates plane hypotheses from mat...

Empirical Inference IS Colloquium 07-04-2014 State-space representation of gaussian processes for regression and efficient inference in latent force model Gaussian process regression is a non-parametric Bayesian machine learning paradigm, where instead of estimating parameters of fixed-form functions, we model the whole unknown functions as Gaussian processes. Gaussian processes are also commonly used for representing uncertainties in models of dynamic systems in many applications such as tracking, navigation, and automatic control systems. The latter models are often formulated as state-space models, where the use of non-linear Kalman filter type of methods is common. The aim of this talk is to discuss connections of Kalman filtering method... Philipp Hennig

Empirical Inference IS Colloquium 31-03-2014 Online Spot Volatility-Estimation and Decomposition with Nonlinear Market Microstructure Noise Models (joint work with Jan. C. Neddermeyer) A technique for online estimation of spot volatility for high-frequency data is developed. The algorithm works directly on the transaction data and updates the volatility estimate immediately after the occurrence of a new transaction. Furthermore, a nonlinear market microstructure noise model is proposed that reproduces several stylized facts of high frequency data. A computationally efficient particle filter is used that allows for the approximation of the unknown efficient prices and, in combination with a recursive EM algorithm, for the estimation ... Michel Besserve

Empirical Inference IS Colloquium 28-03-2014 Simulation in physical scene understanding Our ability to understand a scene is central to how we interact with our environment and with each other. Classic research on visual scene perception has focused on how people "know what is where by looking", but this talk will explore people's ability to infer the "hows" and "whys" of their world, and in particular, how they form a physical understanding of a scene. From a glance we can know so much: not only what objects are where, but whether they are movable, fragile, slimy, or hot; whether they were made by hand, by machine, or by nature; whether they are broken and how they could be r... Michel Besserve

Event 27-03-2014 Girls Day Claudia Daefler

Perceiving Systems Talk 27-03-2014 Video-based Analysis of Humans and Their Behavior This talk will give an overview of some of the research in the Image and Video Computing Group at Boston University related to image- and video-based analysis of humans and their behavior, including: tracking humans, localizing and classifying actions in space-time, exploiting contextual cues in action classification, estimating human pose from images, analyzing the communicative behavior of children in video, and sign language recognition and retrieval. Collaborators in this work include (in alphabetical order): Vassilis Athitsos, Qinxun Bai, Margrit Betke, R. Gokberk Cinbis, Kun He,...

Perceiving Systems IS Colloquium 20-03-2014 Multi-View Perception of Dynamic Scenes <div style="margin: 0px;"> The INRIA MORPHEO research team is working on the perception of moving shapes using multiple camera systems. Such systems allows to recover dense information on shapes and their motions using visual cues. This opens avenues for research investigations on how to model, understand and animate real dynamic shapes using several videos. In this talk I will more particularly focus on recent activities in the team on two fundamental components of the multi-view perception of dynamic scenes that are: (i) the recovery of time-consistent shape models or shape tracki... Gerard Pons-Moll

Perceiving Systems Talk 12-03-2014 Producing free viewpoint 3D video from a real soccer game and its user interface for the virtual camera control This talk presents our 3D video production method by which a user can watch a  real game from any free viewpoint. Players in the game are captured by 10 cameras and they are reproduced three dimensionally by billboard based representation in real time. Upon producing the 3D video, we have also worked on good user interface that can enable people move the camera intuitively. As the speaker is also working on wide variety of computer vision to augmented reality, selected recent works will be also introduced briefly. Dr. Yoshinari K...

Perceiving Systems Talk 11-02-2014 Interactive and Task-driven Multi-view 3D Reconstruction 3D reconstruction from 2D still-images (Structure-from-Motion) has reached maturity and together with new image acquisition devices like Micro Aerial Vehicles (MAV), new interesting application scenarios arise. However, acquiring an image set which is suited for a complete and accurate reconstruction is even for expert users a non-trivial task. To overcome this problem, we propose two different methods. In the first part of the talk, we will present a SfM method that performs sparse reconstruction of 10Mpx still-images and a surface extraction from sparse and noisy 3D point clouds in real-t...

Perceiving Systems IS Colloquium 10-02-2014 Towards Visual Scene Understanding - Articulated Pose Estimation and Video Description This talk will highlight recent progress on two fronts. First, we will talk about a novel image-conditioned person model that allows for effective articulated pose estimation in realistic scenarios. Second, we describe our work towards activity recognition and the ability to describe video content with natural language.  Both efforts are part of a longer-term agenda towards visual scene understanding. While visual scene understanding has long been advocated as the "holy grail" of computer vision, we believe it is time to address this cha...

Perceiving Systems IS Colloquium 13-01-2014 Identity Preserving Multi-People Tracking through Linear Programming In this talk, I will show that, given probabilities of presence of people at various locations in individual time frames, finding the most likely set of trajectories amounts to solving a linear program that depends on very few parameters. This can be done without requiring appearance information and in real-time, by using the K-Shortest Paths algorithm (KSP). However, this can result in unwarranted identity switches in complex scenes. In such cases, sparse image information can be used within the Linear Programming framework to keep track of people's identities, even when...

Perceiving Systems Talk 18-11-2013 Local metric approach in Gaussian Processes Latent Variables Models Manifold learning techniques attempt to map a high-dimensional space onto a lower-dimensional one. From a mathematical point of view, a manifold is a topological Hausdorff space that is locally Euclidean. From Machine Learning point of view, we can interpret this embedded manifold as the underlying support of the data distribution. When dealing with high dimensional data sets, nonlinear dimensionality reduction methods can provide more faithful data representation than linear ones. However, the local geometrical distortion induced by the nonlinear mapping leads to a loss of informatio...

Perceiving Systems Talk 11-11-2013 Perceptual Grouping using Superpixels Perceptual grouping played a prominent role in support of early object recognition systems, which typically took an input image and a database of shape models and identified which of the models was visible in the image.  When the database was large, local features were not sufficiently distinctive to prune down the space of models to a manageable number that could be verified.  However, when causally related shape features were grouped, using intermediate-level shape priors, e.g., cotermination, symmetry, and compactness, they formed effective shape indices and all...

Perceiving Systems Talk 14-10-2013 Inventing the Future at Microsoft Research India: Our first 8 years T.b.a.

Perceiving Systems Talk 11-10-2013 Exploring and editing the appearance of outdoor scenes <div> The appearance of outdoor scenes changes dramatically with lighting and weather conditions, time of day, and season. Specific conditions, such as the "golden hours" characterized by warm light, can be hard to capture because many scene properties are transient -- they change over time. Despite significant advances in image editing software, common image manipulation tasks such as lighting editing require significant expertise to achieve plausible results.</div> <div>  </div> <div> In this talk, we first explore the appearance of outdoor scenes with an approach base...

Max Planck Lecture 10-10-2013 Some Small Steps toward Artificial Life No one has successfully defined life but the properties we often associate with living things are motility, metabolism and self-replication. According to the Nobel Laureate Richard Feynman: "What I can't create, I don't understand". We thought we'd give it a shot – understanding life – and in the process we've made two different systems, one that exhibits both autonomous motility and metabolism and another which is the first artificial system which can replicate arbitrarily designed motifs.

Perceiving Systems Talk 01-10-2013 Inference in highly-connected CRFs This talk presents recent work from CVPR that looks at inference for pairwise CRF models in the highly (or fully) connected case rather than simply a sparse set of neighbours used ubiquitously in many computer vision tasks. Recent work has shown that fully-connected CRFs, where each node is connected to every other node, can be solved very efficiently under the restriction that the pairwise term is a Gaussian kernel over a Euclidean feature space. The method presented generalises this model to allow arbitrary, non-parametric models (which can be learnt from training data and condi...

Perceiving Systems Talk 23-09-2013 Human perception of material properties in the real world Humans are very good at recognizing objects as well as the materials that they are made of. We can easily tell cheese from butter, silk from linen and snow from ice just by looking. Understanding material perception is important for many real-world applications. For instance, a robot cooking in the kitchen will benefit from the knowledge of material perception when deciding if food is cooked or raw. In this talk, I will present studies that are motivated by two important applications of material perception: online shopping and computer graphics (CG) rendering. First, I will discuss the...

Perceiving Systems Talk 23-09-2013 Shape Knowledge in Segmentation and Tracking In this talk I will detail methods for simultaneous 2D/3D segmentation, tracking and reconstruction which incorporate high level shape information. I base my work on the assumption that the space of possible 2D object shapes can be either generated by projecting down known rigid 3D shapes or learned from 2D shape examples. I minimise the discrimination between statistical foreground and background appearance models with respect to the parameters governing the shape generative process (the 6 degree-of-freedom 3D pose of the 3D shape or the parameters of the learned space). The foregroun...

Perceiving Systems Talk 12-09-2013 Efficient Inference and Learning for Structured Parameterizations/Models Sensors acquire an increasing amount of diverse information posing two challenges. Firstly, how can we efficiently deal with such a big amount of data and secondly, how can we benefit from this diversity? In this talk I will first present an approach to deal with large graphical models. The presented method distributes and parallelizes the computation and memory requirements while preserving convergence and optimality guarantees of existing inference and learning algorithms. I will demonstrate the effectiveness of the approach on stereo reconstruction from high-resolution imagery. In the se...

Perceiving Systems Talk 10-09-2013 Depth, You, and the World Consumer level depth cameras such as Kinect have changed the landscape of 3D computer vision.  In this talk we will discuss two approaches that both learn to directly infer correspondences between observed depth image pixels and 3D model points.  These correspondences can then be used to drive an optimization of a generative model to explain the data.  The first approach, the "Vitruvian Manifold", aims to fit an articulated 3D human model to a depth camera image, and extends our original Body Part Recognition algorithm used in Kinect.  It applies a per-pi...

Perceiving Systems Talk 09-09-2013 2D and 3D object detection by exploiting segmentation and contextual information <div> Object detection is one of the main challenges of computer vision. In the standard setting, we are given an image and the goal is to place bounding boxes around the objects and recognize their classes. In robotics, estimating additional information such as accurate viewpoint or detailed segmentation is important for planning and interaction. In this talk, I'll approach detection in three scenarios: purely 2D, 3D from 2D and 3D from 3D and show how different types of information can be used to significantly boost the current state-of-the-art in detection.</div> <div>  </div>

Perceiving Systems Talk 09-09-2013 Efficient Algorithms for Semantic Scene Parsing Developing autonomous systems that are able to assist humans in everyday's tasks is one of the grand challenges in modern computer science. Notable examples are personal robotics for the elderly and people with disabilities, as well as autonomous driving systems which can help decrease fatalities caused by traffic accidents. In order to perform tasks such as navigation, recognition and manipulation of objects, these systems should be able to efficiently extract 3D knowledge of their environment.  In this talk, I'll show how Markov random fields provide a great mathematical form...

Perceiving Systems Talk 14-06-2013 Biomechanical models for animation and life sciences Motion capture and data driven technologies have come very far over the past few years. In terms of human capture the high volume of research that has gone into this sub group has led to very impressive results. Human motion can now be captured in real time which when used in the creative sectors can lead to blockbuster films such as Avatar. Similarly in the medical sectors these techniques can be used to diagnose, analyse performance and avoid invasive procedures in tasks such...

Perceiving Systems Talk 03-06-2013 Discriminative Non-blind Deblurring Non-blind deblurring is an integral component of blind approaches for removing image blur due to camera shake. Even though learning-based deblurring methods exist, they have been limited to the generative case and are computationally expensive. To this date, manually-defined models are thus most widely used, though limiting the attained restoration quality. We address this gap by proposing a discriminative approach for non-blind deblurring. One key challenge is that the blur kernel in use at test time is not known in advance. To address this, we analyze existing approaches that use h...

Perceiving Systems Talk 27-05-2013 Interactive Variational Shape Modeling Irregular triangle meshes are a powerful digital shape representation: they are flexible and can represent virtually any complex shape; they are efficiently rendered by graphics hardware; they are the standard output of 3D acquisition and routinely used as input to simulation software. Yet irregular meshes are difficult to model and edit because they lack a higher-level control mechanism. In this talk, I will survey a series of research results on surface modeling with meshes and show how high-quality shapes can be manipulated in a fast and intuitive manner. I will outline the current...

Perceiving Systems Talk 17-05-2013 3D vision in a changing world 3D reconstruction from images has been a tremendous success-story of computer vision, with city-scale reconstruction now a reality.   However, these successes apply almost exclusively in a static world, where the only motion is that of the camera.  Even with the advent of realtime depth cameras, full 3D modelling of dynamic scenes lags behind the rigid-scene case, and for many objects of interest (e.g. animals moving in natural environments), depth sensing remains challenging.  In this talk, I will discuss a range of recent work in the modelling of nonrigid real-wo...

Max Planck Lecture 08-05-2013 Interface Science Interface Science. The last decade has witnessed explosive growth of research on various oxide heterostructures, and discoveries of exciting new interface phenomena. We may be witnessing the emergence of a new scientific discipline – Interface Science, delineated by a distinct new set of problems, techniques, phenomena, and theoretical concepts.

Perceiving Systems Talk 30-04-2013 Human Pose Estimation from Video and Inertial Sensors Significant progress has been made over the last years in estimating people's shape and motion from video and nonetheless the problem still remains unsolved. This is especially true in uncontrolled environments such as people in the streets or the office where background clutter and occlusions make the problem even more challenging. The goal of our research is to develop computational methods that enable human pose estimation from video and inertial sensors in indoor and outdoor environments. Specifically, I will focus on one of our past projects in which we introduce a hybrid H...

Perceiving Systems Talk 29-04-2013 What Make Big Visual Data Hard? There are an estimated 3.5 trillion photographs in the world, of which 10% have been taken in the past 12 months. Facebook alone reports 6 billion photo uploads per month. Every minute, 72 hours of video are uploaded to YouTube. Cisco estimates that in the next few years, visual data (photos and video) will account for over 85% of total internet traffic. Yet, we currently lack effective computational methods for making sense of all this mass of visual data. Unlike easily indexed content, such as text, visual content is not routinely searched or mined; it's not even hyperlinked. Vis...

Perceiving Systems Talk 04-03-2013 Novel design principles for interfacing biological and artificial vision for assistive perceiving systems Studying the interface between artificial and biological vision has been an area of research that has been greatly promoted for a long time. It seems promising that cognitive science can provide new ideas to interface computer vision and human perception, yet no established design principles do exist. In the first part of my talk I am going to introduce the novel concept of 'object detectability'. Object detectability refers to a measure of how likely a human observer is visually aware of the location and presence of specific object types in a complex, dynamic, urban scene....

Perceiving Systems Talk 28-02-2013 Learning a Confidence Measure for Optical Flow We present a supervised learning based method to estimate a per-pixel confidence for optical flow vectors. Regions of low texture and pixels close to occlusion boundaries are known to be difficult for optical flow algorithms. Using a spatiotemporal feature vector, we estimate if a flow algorithm is likely to fail in a given region. Our method is not restricted to any specific class of flow algorithm, and does not make any scene specific assumptions. By automatically learning this confidence we can combine the output of several computed flow fields from different algorithms to...

Perceiving Systems Talk 27-02-2013 Partial Supervision and latent variable models for semantic image segmentation Semantic image segmentation is the task of assigning semantic labels to the pixels of a natural image. It is an important step towards general scene understanding and has lately received much attention in the computer vision community. It was found that detailed annotation of images are helpful for solving this task, but obtaining accurate and consistent annotations still proves to be difficult on a large scale. One possible way forward is to work with partial supervision and latent variable models to infer semantic annotations from the data during training. The talk will pre...

Perceiving Systems Talk 25-02-2013 From Particle Stereo to Scene Stereo In this talk I will present two lines of research which are both applied to the problem of stereo matching. The first line of research tries to make progress on the very traditional problem of stereo matching. In BMVC 11 we presented the PatchmatchStereo work which achieves surprisingly good results with a simple energy function consisting of unary terms only. As optimization engine we used the PatchMatch method, which was designed for image editing purposes. In BMVC 12 we extended this work by adding to the energy function the standard pairwise smoothness terms. The main contribution...

Perceiving Systems Talk 22-01-2013 Large-scale and weakly supervised learning of objects and actions We, first, address the problems of large scale image classification. We present and evaluate different ways of aggregating local image descriptors into a vector and show that the Fisher kernel achieves better performance than the reference bag-of-visual words approach for any given vector dimension. We show and interpret the importance of an appropriate vector normalization. Furthermore, we discuss how to learn given a large number of classes and images with stochastic gradient descent and show results on ImageNet10k. We, then, present a weakly supervised approach for learnin...

Perceiving Systems Talk 22-01-2013 Lie Shapes and Statistics on Manifolds Three-dimensional object shape is commonly represented in terms of deformations of a triangular mesh from an exemplar shape. In particular, statistical generative models of human shape deformation are widely used in computer vision, graphics, ergonomics, and anthropometry. Existing statistical models, however, are based on a Euclidean representation of shape deformations. In contrast, we argue that shape has a manifold structure: For example, averaging the shape deformations for two people does not necessarily yield a meaningful shape deformation, nor does the Euclidean difference of t...

Max Planck Lecture 06-12-2012 Following Function in Real Time: New NMR, MRI and Diffraction Methods for Studying Structure and Dynamics in Batteries and Supercapacitors A full understanding of the operation of a device requires that we utilize methods that allow devices or materials to be probed while they are operating (i.e., in-situ). This allows, for example, the transformations of the various cell components to be followed under realistic conditions without having to disassemble and take apart the cell.

Perceiving Systems Talk 28-11-2012 Whole is more than the Sum of its Parts: Applications to Object Detection and Beyond The grand goal of Computer Vision is to generate an automatic description of an image based on its visual content. Category level object detection is an important building block towards such capability. The first part of this talk deals with three established object detection techniques in Computer Vision, their shortcomings and how they are improved. i) Hough Voting methods efficiently handle the high complexity of multi-scale, category-level object detection in cluttered scenes. However, the primary weakness of this approach is that mutually dependent local observations...

Perceiving Systems Talk 05-11-2012 3D Scene Understanding for Autonomous Vehicles Navigating a car safely through complex environments is considered a relatively easy task for humans. Computer algorithms, however, can't nearly match human performance and often rely on 3D laser scanners or detailed maps. The reason for this is that the level and accuracy of current computer vision and scene understanding algorithms is still far from that of a human being. In this talk I will argue that pushing these limits requires solving a set of core computer vision problems, ranging from low-level tasks (stereo, optical flow) to high-level problems (object detection, 3D scen...

Max Planck Lecture 26-09-2012 Progress on biologically-inspired microrobots As the characteristic size of a flying robot decreases, the challenges for successful flight revert to basic questions of fabrication, actuation, fluid mechanics, stabilization, and power – whereas such questions have in general been answered for larger aircraft.

Perceiving Systems Talk 24-09-2012 How well do filter-based MRFs model natural images? Markov random fields (MRFs) have found widespread use as models of natural image and scene statistics. Despite progress in modeling image properties beyond gradient statistics with high-order cliques, and learning image models from example data, existing MRFs only exhibit a limited ability of actually capturing natural image statistics. In this talk I will present recent work that investigates this limitation of previous filter-based MRF models, including Fields of Experts (FoEs). We found that these limitations are due to inadequacies in the leaning procedure and suggest v...

Perceiving Systems Talk 17-09-2012 What You See is Less Than What You Get - Estimating Visually Non-Observable Object Properties The great majority of object analysis methods are based on visual object properties - objects are categorized according to how they appear in images. Visual appearance is measured in terms of image features (e.g., SIFTs) extracted from images or video. However, besides appearance, objects also have many properties that can be of interest, e.g., for a robot who wants to employ them in activities: Temperature, weight, surface softness, and also the functionalities or affordances of the object, i.e., how it is intended to be used. One example, recently addressed in the vision community, a...

Perceiving Systems Talk 17-07-2012 A Framework for Joint Registration and Statistical Shape Analysis of 2D and 3D Objects Shape analysis and modeling of 2D and 3D objects has important applications in many branches of science and engineering. The general goals in shape analysis include: derivation of efficient shape metrics, computation of shape templates, representation of dominant shape variability in a shape class, and development of probability models that characterize shape variation within and across classes. While past work on shape analysis is dominated by point representations -- finite sets of ordered or triangulated points on objects' boundaries -- the emphasis has lately shifted to contin...

Perceiving Systems Talk 13-07-2012 An approach to light microscopy yielding SEM-like images and 3D surface topography using an elastomeric gelVenue: Werner Köster-Hörsaal (2R4), MPI-IS Stuttgart, Heisenbergstr. 3 Date: Friday, July 13, 2012, 10:30 h Abstract: We can modify the optical properties of surfaces by “coating” them with a micron-thin membrane supported by an elastomeric gel. Using an opaque, matte membrane, we can make reflected light micrographs with a distinctive SEM-like appearance. These have modest magnification (e.g., 50X), but they reveal fine surface details not normally seen with an optical microscope. The system, which we call “GelSight,” removes optical complexities such as specular reflection, albedo, and subsurface scattering, and isolates the shading information that signals 3D sh...

Perceiving Systems Talk 12-07-2012 Lines, shading, and the perception of 3D shape Human can easily see 3D shape from single 2D images, exploiting multiple kinds of information. This has given rise to multiple subfields (in both human vision and computer vision) devoted to the study of shape-from-shading, shape-from-texture, shape-from-contours, and so on. The proposed algorithms for each type of shape-from-x remain specialized and fragile (in contrast with the flexibility and robustness of human vision). Recent work in graphics and psychophysics has demonstrated the importance of local orientation structure in conveying 3D shape. This information is fairl...

Perceiving Systems Talk 12-07-2012 Probing nonlinear computations in the retina at single cell resolutionBernstein lecture, 12 July, 18:15, Children's Hospital. A central aspect of visual processing in the retina is the existence of nonlinear subunits within the receptive fields of retinal ganglion cells. These subunits have been implicated in visual computations such as segregation of object motion from background motion. However, relatively little is known about the spatial structure of subunits and its emergence from nonlinear interactions in the interneuron circuitry of the retina. We used physiological measurements of functional circuitry in the isolated primate retina at single-cell resolution, combined with novel computati...

Perceiving Systems Talk 11-07-2012 What is the representation in early vision? Considerable research has demonstrated that the representation is not equally faithful throughout the visual field; representation appears to be coarser in peripheral vision, perhaps as a strategy for dealing with an information bottleneck in visual processing. In the last few years, a convergence of evidence has suggested that in peripheral and unattended regions, the information available consists of summary statistics. For a complex set of statistics, such a representation can provide a rich and detailed percept of many aspects of a visual scene. However, such a representa...

Research

Departments

Research Groups

People

Contact

Our Institute

Our History

Career

Doctoral Programs

Training

Service Units

Central Scientific Facilities

Workshops

Campus Services

Impact

Cooperation

Partners and Initiatives