Empirical Inference Technical Report 2007

Learning with Transformation Invariant Kernels

Abstract. This paper considers kernels invariant to translation, rotation and dilation. We show that no non-trivial positive definite (p.d.) kernels exist which are radial and dilation invariant, only conditionally positive definite (c.p.d.) ones. Accordingly, we discuss the c.p.d. case and provide some novel analysis, including an elementary derivation of a c.p.d. representer theorem. On the practical side, we give a support vector machine (s.v.m.) algorithm for arbitrary c.p.d. kernels. For the thin-plate kernel this leads to a classifier with only one parameter (the amount of regularisation), which we demonstrate to be as effective as an s.v.m. with the Gaussian kernel, even though the Gaussian involves a second parameter (the length scale).

Author(s): Walder, C. and Chapelle, O.
Number (issue): 165
Year: 2007
Month: September
Day: 0
Bibtex Type: Technical Report (techreport)
Digital: 0
Electronic Archiving: grant_archive
Institution: Max Planck Institute for Biological Cybernetics, Tübingen, Germany
Language: en
Organization: Max-Planck-Gesellschaft
School: Biologische Kybernetik
Links:

BibTex

@techreport{4737,
  title = {Learning with Transformation Invariant Kernels},
  abstract = {Abstract. This paper considers kernels invariant to translation, rotation and dilation. We show that no non-trivial
  positive definite (p.d.) kernels exist which are radial and dilation invariant, only conditionally positive definite
  (c.p.d.) ones. Accordingly, we discuss the c.p.d. case and provide some novel analysis, including an elementary
  derivation of a c.p.d. representer theorem. On the practical side, we give a support vector machine (s.v.m.) algorithm
  for arbitrary c.p.d. kernels. For the thin-plate kernel this leads to a classifier with only one parameter (the
  amount of regularisation), which we demonstrate to be as effective as an s.v.m. with the Gaussian kernel, even
  though the Gaussian involves a second parameter (the length scale).},
  number = {165},
  organization = {Max-Planck-Gesellschaft},
  institution = {Max Planck Institute for Biological Cybernetics, Tübingen, Germany},
  school = {Biologische Kybernetik},
  month = sep,
  year = {2007},
  slug = {4737},
  author = {Walder, C. and Chapelle, O.},
  month_numeric = {9}
}