Empirical Inference Technical Report 2006

Cross-Validation Optimization for Structured Hessian Kernel Methods

We address the problem of learning hyperparameters in kernel methods for which the Hessian of the objective is structured. We propose an approximation to the cross-validation log likelihood whose gradient can be computed analytically, solving the hyperparameter learning problem efficiently through nonlinear optimization. Crucially, our learning method is based entirely on matrix-vector multiplication primitives with the kernel matrices and their derivatives, allowing straightforward specialization to new kernels or to large datasets. When applied to the problem of multi-way classification, our method scales linearly in the number of classes and gives rise to state-of-the-art results on a remote imaging task.

Author(s): Seeger, M. and Chapelle, O.
Year: 2006
Month: February
Day: 0
Bibtex Type: Technical Report (techreport)
Digital: 0
Electronic Archiving: grant_archive
Institution: Max-Planck Institute for Biological Cybernetics, Tübingen, Germany
Language: en
Organization: Max-Planck-Gesellschaft
School: Biologische Kybernetik
Links:

BibTex

@techreport{3863,
  title = {Cross-Validation Optimization for Structured Hessian Kernel Methods},
  abstract = {We address the problem of learning hyperparameters in kernel methods for
  which the Hessian of the objective is structured. We propose an approximation
  to the cross-validation log likelihood whose gradient can be computed
  analytically, solving the hyperparameter learning problem efficiently
  through nonlinear optimization. Crucially, our learning method is based
  entirely on matrix-vector multiplication primitives with the kernel
  matrices and their derivatives, allowing straightforward specialization to
  new kernels or to large datasets. When applied to the problem of multi-way
  classification, our method scales linearly in the number of classes and
  gives rise to state-of-the-art results on a remote imaging task.},
  organization = {Max-Planck-Gesellschaft},
  institution = {Max-Planck Institute for Biological Cybernetics, Tübingen, Germany},
  school = {Biologische Kybernetik},
  month = feb,
  year = {2006},
  slug = {3863},
  author = {Seeger, M. and Chapelle, O.},
  month_numeric = {2}
}