Events & Talks

Empirical Inference Talk 11-04-2019 Unsupervised Learning: Passiv and Active I’ll start with a concept of 1990 that has become popular: unsupervised learning without a teacher through two adversarial neural networks (NNs) that duel in a minimax game, where one NN minimizes the objective function maximized by the other. The first NN generates data through its output actions, the second NN predicts the data. The second NN minimizes its error, thus becoming a better predictor. But it is a zero sum game: the first NN tries to find actions that maximize the error of the second NN. The system exhibits what I called “artificial curiosity” because the first NN is motivated ... Bernhard Schölkopf

Empirical Inference IS Colloquium 25-03-2019 Probabilistic symmetry and invariant neural networks In an effort to improve the performance of deep neural networks in data-scarce, non-i.i.d., or unsupervised settings, much recent research has been devoted to encoding invariance under symmetry transformations into neural network architectures. We treat the neural network input and output as random variables, and consider group invariance from the perspective of probabilistic symmetry. Drawing on tools from probability and statistics, we establish a link between functional and probabilistic symmetry, and obtain functional representations of probability distributions that are invariant or eq... Isabel Valera

Empirical Inference Talk 11-06-2018 Double Machine Learning with two Applications: Transformation Models and Gaussian Graphical Models in High-Dimensional Settings In this talk first an introduction to the double machine learning framework is given. This allows inference on parameters in high-dimensional settings. Then, two applications are given, namely transformation models and Gaussian graphical models in high-dimensional settings. Both kind of models are widely used by practitioners. As high-dimensional data sets become more and more available, it is important to allow situations where the number of parameters is large compared to the sample size. Philipp Geiger

Empirical Inference IS Colloquium 11-06-2018 Learning Representations for Hyperparameter Transfer Learning Bayesian optimization (BO) is a model-based approach for gradient-free black-box function optimization, such as hyperparameter optimization. Typically, BO relies on conventional Gaussian process regression, whose algorithmic complexity is cubic in the number of evaluations. As a result, Gaussian process-based BO cannot leverage large numbers of past function evaluations, for example, to warm-start related BO runs. After a brief intro to BO and an overview of several use cases at Amazon, I will discuss a multi-task adaptive Bayesian linear regression model, whose computational complexity is ... Isabel Valera

Empirical Inference Talk 14-05-2018 Machine learning in cognitive neuroscience: questions, challenges and potential opportunities In this talk I will describe the main types of research questions and neuroimaging tools used in my work in human cognitive neuroscience (with foci in audition and sleep), some of the existing approaches used to analyze our data, and their limitations. I will then discuss the main practical obstacles to applying machine learning methods in our field. Several of my ongoing and planned projects include research questions that could be addressed and perhaps considerably extended using machine learning approaches; I will describe some specific datasets and problems, with the goal of exploring i... Mara Cascianelli

Empirical Inference IS Colloquium 23-03-2018 The Impact of Big Data on Firm Performance: an Empirical Investigation In academic and policy circles, there has been considerable interest in the impact of “big data” on firm performance. We examine the question of how the amount of data impacts the accuracy of Machine Learned models of weekly retail product forecasts using a proprietary data set obtained from Amazon. We examine the accuracy of forecasts in two relevant dimensions: the number of products (N), and the number of time periods for which a product is available for sale (T). Theory suggests diminishing returns to larger N and T, with relative forecast errors diminishing at rate 1/sqrt(N) + 1/sqrt(... Michel Besserve Michael Hirsch

Empirical Inference IS Colloquium 05-03-2018 Three principles of data science: predictability, stability, and computability In this talk, I'd like to discuss the intertwining importance and connections of three principles of data science in the title. They will be demonstrated in the context of two collaborative projects in neuroscience and genomics, respectively. The first project in neuroscience uses transfer learning to integrate fitted convolutional neural networks (CNNs) on ImageNet with regression methods to provide predictive and stable characterizations of neurons from the challenging primary visual cortex V4. The second project proposes iterative random forests (iRF) as a stablized RF to seek predictab... Michel Besserve

Empirical Inference Talk 27-02-2018 A naturalistic perspective on optic flow processing in the fly Optic flow offers a rich source of information about an organism’s environment. Flies, for instance, are thought to make use of motion vision to control and stabilise their course during acrobatic airborne manoeuvres. How these computations are implemented in neural hardware and how such circuits cope with the visual complexity of natural scenes, however, remain open questions. This talk outlines some of the progress we have made in unraveling the computational substrate underlying optic flow processing in Drosophila. In particular, I will focus on our efforts to connect neural mechanisms a... Michel Besserve

Empirical Inference IS Colloquium 23-10-2017 Modern Optimization for Structured Machine Learning Machine learning has become a popular application domain for modern optimization techniques, pushing its algorithmic frontier. The need for large scale optimization algorithms which can handle millions of dimensions or data points, typical for the big data era, have brought a resurgence of interest for first order algorithms, making us revisit the venerable stochastic gradient method [Robbins-Monro 1951] as well as the Frank-Wolfe algorithm [Frank-Wolfe 1956]. In this talk, I will review recent improvements on these algorithms which can exploit the structure of modern machine learning appro... Philipp Hennig

Empirical Inference IS Colloquium 02-10-2017 Algorithms for survival: a decision-theoretic perspective on adaptive action under threat Under acute threat, biological agents need to choose adaptive actions to survive. In my talk, I will provide a decision-theoretic view on this problem and ask, what are potential computational algorithms for this choice, and how are they implemented in neural circuits. Rational design principles and non-human animal data tentatively suggest a specific architecture that heavily relies on tailored algorithms for specific threat scenarios. Virtual reality computer games provide an opportunity to translate non-human animal tasks to humans and investigate these algorithms across species. I will ... Michel Besserve

Empirical Inference Talk 25-07-2017 Low-dimensional Data Embedding via Robust Ranking Bernhard Schölkopf

Empirical Inference Talk 24-07-2017 Adaptive Learning Rate Algorithms for Stochastic Optimization and Variational Bayesian Inference We present a way to set the step size of Stochastic Gradient Descent, as the solution of a distance minimization problem. The obtained result has an intuitive interpretation and resembles the update rules of well known optimization algorithms. Also, asymptotic results to its relation to the optimal learning rate of Gradient Descent are discussed. In addition, we talk about two different estimators, with applications in Variational inference problems, and present approximate results about their variance. Finally, we combine all of the above, to present an optimization algorithm that... Philipp Hennig

Empirical Inference Talk 11-07-2017 Soft bioelectronics: Materials and Technology Bioelectronics integrates principles of electrical engineering and materials science to biology, medicine and ultimately health. Soft bioelectronics focus on designing and manufacturing electronic devices with mechanical properties close to those of the host biological tissue so that long-term reliability and minimal perturbation are induced in vivo and/or truly wearable systems become possible. We illustrate the potential of this soft technology with examples ranging from prosthetic tactile skins to soft multimodal neural implants.

Empirical Inference Talk 10-07-2017 Sentiment analysis of tweets to detect tipping points in vaccinating behaviour Vaccine refusal can lead to outbreaks of previously eradicated diseases and is an increasing problem worldwide. Vaccinating decisions exemplify a complex, coupled system where vaccinating behavior and disease dynamics influence one another. Complex systems often exhibit characteristic dynamics near a tipping point to a new dynamical regime. For instance, critical slowing down -- the tendency for a system to start `wobbling'-- can increase close to a tipping point. We used a linear support vector machine to classify the sentiment of geo-located United States and California tweets concern...

Empirical Inference IS Colloquium 03-07-2017 Causal Macro Variables Standard methods of causal discovery take as input a statistical data set of measurements of well-defined causal variables. The goal is then to determine the causal relations among these variables. But how are these causal variables identified or constructed in the first place? Often we have sensor level data but assume that the relevant causal interactions occur at a higher scale of aggregation. Sometimes we only have aggregate measurements of causal interactions at a finer scale. I will motivate the general problem of causal discovery and present recent work on a framework and meth... Sebastian Weichwald

Empirical Inference Talk 13-06-2017 Model-based reinforcement learning for sequential decision-making Autonomous systems rely on learning from experience to automatically refine their strategy and adapt to their environment, and thereby have huge advantages over traditional hand engineered systems. At PROWLER.io we use reinforcement learning (RL) for sequential decision making under uncertainty to develop intelligent agents capable of acting in dynamic and unknown environments. In this talk we first give a general overview of the goals and the research conducted at PROWLER.io. Then, we will talk about two specific research topics. The first is Information-Theoretic Model Uncertainty which d... Michel Besserve

Empirical Inference IS Colloquium 29-05-2017 Probabilistic Deep Learning: From Density Estimation to Representation Learning Probabilistic deep learning methods have recently made great progress for generative and discriminative modeling. I will give a brief overview of recent developments and then present two contributions. The first is on a generalization of generative adversarial networks (GAN), extending their use considerably. GANs can be shown to approximately minimize the Jensen-Shannon divergence between two distributions, the true sampling distribution and the model distribution. We extend GANs to the class of f-divergences which include popular divergences such as the Kullback-Leibler diver... Lars Mescheder

Empirical Inference IS Colloquium 06-03-2017 Statistical testing of epiphenomena for multi-index data As large tensor-variate data increasingly become the norm in applied machine learning and statistics, complex analysis methods similarly increase in prevalence. Such a trend offers the opportunity to understand more intricate features of the data that, ostensibly, could not be studied with simpler datasets or simpler methodologies. While promising, these advances are also perilous: these novel analysis techniques do not always consider the possibility that their results are in fact an expected consequence of some simpler, already-known feature of simpler data (for example, treating the te... Philipp Hennig

Empirical Inference IS Colloquium 19-12-2016 Human Learning and Alternative Applications Towards Usable Electroencephalography-based Brain-Computer Interfaces Brain-Computer Interfaces (BCIs) are systems that can translate brain activity patterns of a user into messages or commands for an interactive application. Such brain activity is typically measured using Electroencephalography (EEG), before being processed and classified by the system. EEG-based BCIs have proven promising for a wide range of applications ranging from communication and control for motor impaired users, to gaming targeted at the general public, real-time mental state monitoring and stroke rehabilitation, to name a few. Despite this promising potential, BCIs are still scarcely...

Empirical Inference Talk 01-03-2016 Images of planets orbiting other stars The detection and characterization of planets orbiting other stars than the Sun, i.e., so-called extrasolar planets, is one of the fastest growing and most vibrant research fields in modern astrophysics. In the last 25 years, more than 5400 extrasolar planets and planet candidates were revealed, but the vast majority of these objects was detected with indirect techniques, where the existence of the planet is inferred from periodic changes in the light coming from the central star. No photons from the planets themselves are detected. In this talk, however, I will focus on the direct detectio...

Empirical Inference IS Colloquium 25-01-2016 What can we learn if we record the complete perception and action of a person? Our research questions are centred on a basic characteristic of human brains: variability in their behaviour and their underlying meaning for cognitive mechanisms. Such variability is emerging as a key ingredient in understanding biological principles (Faisal, Selen & Wolpert, 2008, Nature Rev Neurosci) and yet lacks adequate quantitative and computational methods for description and analysis. Crucially, we find that biological and behavioural variability contains important information that our brain and our technology can make us of (instead of just averaging it away): Using advanced body... Matthias Hohmann

Empirical Inference IS Colloquium 19-10-2015 Imaging genomics of functional brain networks During rest, brain activity is intrinsically synchronized between different brain regions, forming networks of coherent activity. These functional networks (FNs), consisting of multiple regions widely distributed across lobes and hemispheres, appear to be a fundamental theme of neural organization in mammalian brains. Despite hundreds of studies detailing this phenomenon, the genetic and molecular mechanisms supporting these functional networks remain undefined. Previous work has mostly focused on polymorphisms in candidate genes, or used a twin study approach to demonstrate heritability... Moritz Grosse-Wentrup Michel Besserve

Empirical Inference Talk 09-10-2015 Using machine learning to understand cognition from brain images Moritz Grosse-Wentrup

Empirical Inference IS Colloquium 28-09-2015 High-dimensional statistical approaches for personalized medicine Human diseases show considerable heterogeneity at the molecular level. Such heterogeneity is central to personalized medicine efforts that seek to exploit molecular data to better understand disease biology and inform clinical decision making. An emerging notion is that diseases and disease subgroups may differ not only at the level of mean molecular abundance, but also with respect to patterns of molecular interplay. I will discuss our ongoing efforts to develop methods to investigate such heterogeneity, with an emphasis on some high-dimensional aspects. Michel Besserve Jonas Peters

Empirical Inference IS Colloquium 27-07-2015 New Foundations for Ockham’s Razor in Statistical Inductive Inference, with Applications to Causal Discovery from Non-experimental Data In machine learning, the standard explanation of Ockham's razor is to minimize predictive risk. But prediction is interpreted passively---one may not rely on predictions to change the probability distribution used for training. That limitation may be overcome by studying alternatively manipulated systems in randomized experimental trials, but experiments on multivariate systems or on human subjects are often infeasible or immoral. Happily, the past three decades have witnessed the development of a range of statistical techniques for discovering causal relations from non-experimental dat... Michel Besserve Kun Zhang

Empirical Inference Conference 13-07-2015 - 24-07-2015 MLSS 2015 - Machine Learning Summer School 2015 in Tübingen MLSS's are a renowned venue for graduate students, researchers, and professionals. They offer an opportunity to learn about fundamental and advanced aspects of machine learning, data analysis and inference, from intellectual leaders of the field. <br> In the summer of 2015 the MLSS will make its fourth appearance in Tübingen. Michael Hirsch Philipp Hennig Bernhard Schölkopf

Empirical Inference IS Colloquium 11-08-2014 Examples of Machine Learning and Data Science at Facebook Facebook serves close to a billion people every day, who are only able to consume a small subset of the information available to them. In this talk I will give some examples of how machine learning is used to personalize people’s Facebook experience. I will also present some data science experiments with fairly counter-intuitive results.

Empirical Inference IS Colloquium 15-07-2014 Approximate inference for stochastic differential equations Stochastic differential equations (SDEs) arise naturally as descriptions of continuous time dynamical systems. My talk addresses the problem of inferring the dynamical state and parameters of such systems from observations taken at discrete times. I will discuss the application of approximate inference methods such as the variational method and expectation propagation and show how higher dimensional systems can be treated by a mean field approximation. In the second part of my talk I will discuss the nonparametric estimation of the drift (i.e. the deterministic part of the ‘force’ whi... Philipp Hennig Michel Besserve

Empirical Inference IS Colloquium 23-06-2014 Compressive Sensing and Beyond The recent theory of compressive sensing predicts that (approximately) sparse vectors can be recovered from vastly incomplete linear measurements using efficient algorithms. This principle has a large number of potential applications in signal and image processing, machine learning and more. Optimal measurement matrices in this context known so far are based on randomness. Recovery algorithms include convex optimization approaches (l1-minimization) as well as greedy methods. Gaussian and Bernoulli random matrices are provably optimal in the sense that the smallest possible number of... Michel Besserve

Empirical Inference Symposium 04-06-2014 - 06-06-2014 IEEE Conference on Pattern Recognition in Neuroimaging, Tübingen 2014 Multivariate analysis of neuroimaging data has gained ground very rapidly in the community over the past few years, leading to impressive results in cognitive and clinical neuroscience. Pattern recognition and machine learning conferences regularly feature a neuroimaging workshop, while neuroscientific meetings dedicate sessions to new approaches to neural data analysis. Thus, a rich two-way flow has been established between disciplines. It is the goal of the PRNI workshop series to continue facilitating exchange of ideas between scientific communities, with a particular interest in new app... Moritz Grosse-Wentrup

Empirical Inference IS Colloquium 26-05-2014 Low-rank dynamics This talk reviews differential equations on manifolds of matrices or tensors of low rank. They serve to approximate, in a low-rank format, large time-dependent matrices and tensors that are either given explicitly via their increments or are unknown solutions of differential equations. Furthermore, low-rank differential equations are used in novel algorithms for eigenvalue optimisation, for instance in robust-stability problems. Philipp Hennig

Empirical Inference IS Colloquium 07-04-2014 State-space representation of gaussian processes for regression and efficient inference in latent force model Gaussian process regression is a non-parametric Bayesian machine learning paradigm, where instead of estimating parameters of fixed-form functions, we model the whole unknown functions as Gaussian processes. Gaussian processes are also commonly used for representing uncertainties in models of dynamic systems in many applications such as tracking, navigation, and automatic control systems. The latter models are often formulated as state-space models, where the use of non-linear Kalman filter type of methods is common. The aim of this talk is to discuss connections of Kalman filtering method... Philipp Hennig

Empirical Inference IS Colloquium 31-03-2014 Online Spot Volatility-Estimation and Decomposition with Nonlinear Market Microstructure Noise Models (joint work with Jan. C. Neddermeyer) A technique for online estimation of spot volatility for high-frequency data is developed. The algorithm works directly on the transaction data and updates the volatility estimate immediately after the occurrence of a new transaction. Furthermore, a nonlinear market microstructure noise model is proposed that reproduces several stylized facts of high frequency data. A computationally efficient particle filter is used that allows for the approximation of the unknown efficient prices and, in combination with a recursive EM algorithm, for the estimation ... Michel Besserve

Empirical Inference IS Colloquium 28-03-2014 Simulation in physical scene understanding Our ability to understand a scene is central to how we interact with our environment and with each other. Classic research on visual scene perception has focused on how people "know what is where by looking", but this talk will explore people's ability to infer the "hows" and "whys" of their world, and in particular, how they form a physical understanding of a scene. From a glance we can know so much: not only what objects are where, but whether they are movable, fragile, slimy, or hot; whether they were made by hand, by machine, or by nature; whether they are broken and how they could be r... Michel Besserve

Research

Departments

Research Groups

People

Contact

Our Institute

Our History

Career

Doctoral Programs

Training

Service Units

Central Scientific Facilities

Workshops

Campus Services

Impact

Cooperation

Partners and Initiatives