Empirical Inference Article 2008

Approximations for Binary Gaussian Process Classification

We provide a comprehensive overview of many recent algorithms for approximate inference in Gaussian process models for probabilistic binary classification. The relationships between several approaches are elucidated theoretically, and the properties of the different algorithms are corroborated by experimental results. We examine both 1) the quality of the predictive distributions and 2) the suitability of the different marginal likelihood approximations for model selection (selecting hyperparameters) and compare to a gold standard based on MCMC. Interestingly, some methods produce good predictive distributions although their marginal likelihood approximations are poor. Strong conclusions are drawn about the methods: The Expectation Propagation algorithm is almost always the method of choice unless the computational budget is very tight. We also extend existing methods in various ways, and provide unifying code implementing all approaches.

Author(s): Nickisch, H. and Rasmussen, CE.
Journal: Journal of Machine Learning Research
Volume: 9
Pages: 2035-2078
Year: 2008
Month: October
Day: 0
Bibtex Type: Article (article)
Digital: 0
Electronic Archiving: grant_archive
Language: en
Organization: Max-Planck-Gesellschaft
School: Biologische Kybernetik
Links:

BibTex

@article{5305,
  title = {Approximations for Binary Gaussian Process Classification},
  journal = {Journal of Machine Learning Research},
  abstract = {We provide a comprehensive overview of many recent algorithms for approximate inference in
  Gaussian process models for probabilistic binary classification. The relationships between several
  approaches are elucidated theoretically, and the properties of the different algorithms are
  corroborated by experimental results. We examine both 1) the quality of the predictive distributions and
  2) the suitability of the different marginal likelihood approximations for model selection (selecting
  hyperparameters) and compare to a gold standard based on MCMC. Interestingly, some methods
  produce good predictive distributions although their marginal likelihood approximations are poor.
  Strong conclusions are drawn about the methods: The Expectation Propagation algorithm is almost
  always the method of choice unless the computational budget is very tight. We also extend
  existing methods in various ways, and provide unifying code implementing all approaches.},
  volume = {9},
  pages = {2035-2078},
  organization = {Max-Planck-Gesellschaft},
  school = {Biologische Kybernetik},
  month = oct,
  year = {2008},
  slug = {5305},
  author = {Nickisch, H. and Rasmussen, CE.},
  month_numeric = {10}
}