Empirical Inference Article 2008

Kernels, Regularization and Differential Equations

Many common machine learning methods such as Support Vector Machines or Gaussian process inference make use of positive definite kernels, reproducing kernel Hilbert spaces, Gaussian processes, and regularization operators. In this work these objects are presented in a general, unifying framework, and interrelations are highlighted. With this in mind we then show how linear stochastic differential equation models can be incorporated naturally into the kernel framework. And vice versa, many kernel machines can be interpreted in terms of differential equations. We focus especially on ordinary differential equations, also known as dynamical systems, and it is shown that standard kernel inference algorithms are equivalent to Kalman filter methods based on such models. In order not to cloud qualitative insights with heavy mathematical machinery, we restrict ourselves to finite domains, implying that differential equations are treated via their corresponding finite difference equations.

Author(s): Steinke, F. and Schölkopf, B.
Journal: Pattern Recognition
Volume: 41
Number (issue): 11
Pages: 3271-3286
Year: 2008
Month: November
Day: 0
Bibtex Type: Article (article)
DOI: 10.1016/j.patcog.2008.06.011
Digital: 0
Electronic Archiving: grant_archive
Language: en
Organization: Max-Planck-Gesellschaft
School: Biologische Kybernetik
Links:

BibTex

@article{5251,
  title = {Kernels, Regularization and Differential Equations},
  journal = {Pattern Recognition},
  abstract = {Many common machine learning methods such as Support Vector Machines or Gaussian process
  inference make use of positive definite kernels, reproducing kernel Hilbert spaces, Gaussian processes, and
  regularization operators. In this work these objects are presented in a general, unifying framework, and
  interrelations are highlighted.
  With this in mind we then show how linear stochastic differential equation models can be incorporated
  naturally into the kernel framework. And vice versa, many kernel machines can be interpreted in terms of
  differential equations. We focus especially on ordinary differential equations, also known as dynamical
  systems, and it is shown that standard kernel inference algorithms are equivalent to Kalman filter methods
  based on such models.
  In order not to cloud qualitative insights with heavy mathematical machinery, we restrict ourselves to finite
  domains, implying that differential equations are treated via their corresponding finite difference equations.},
  volume = {41},
  number = {11},
  pages = {3271-3286},
  organization = {Max-Planck-Gesellschaft},
  school = {Biologische Kybernetik},
  month = nov,
  year = {2008},
  slug = {5251},
  author = {Steinke, F. and Sch{\"o}lkopf, B.},
  month_numeric = {11}
}