Empirische Inferenz Conference Paper 2008

Semi-Supervised Laplacian Regularization of Kernel Canonical Correlation Analysis

Kernel canonical correlation analysis (KCCA) is a dimensionality reduction technique for paired data. By finding directions that maximize correlation, KCCA learns representations that are more closely tied to the underlying semantics of the data rather than noise. However, meaningful directions are not only those that have high correlation to another modality, but also those that capture the manifold structure of the data. We propose a method that is simultaneously able to find highly correlated directions that are also located on high variance directions along the data manifold. This is achieved by the use of semi-supervised Laplacian regularization of KCCA. We show experimentally that Laplacian regularized training improves class separation over KCCA with only Tikhonov regularization, while causing no degradation in the correlation between modalities. We propose a model selection criterion based on the Hilbert-Schmidt norm of the semi-supervised Laplacian regularized cross-covariance operator, which we compute in closed form.

Author(s): Blaschko, MB. and Lampert, CH. and Gretton, A.
Book Title: ECML PKDD 2008
Journal: Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2008
Pages: 133-145
Year: 2008
Month: August
Day: 0
Editors: Daelemans, W. , B. Goethals, K. Morik
Publisher: Springer
Bibtex Type: Conference Paper (inproceedings)
Address: Berlin, Germany
DOI: 10.1007/978-3-540-87479-9_27
Event Name: 19th European Conference on Machine Learning
Event Place: Antwerpen, Belgium
Digital: 0
Electronic Archiving: grant_archive
Language: en
Organization: Max-Planck-Gesellschaft
School: Biologische Kybernetik
Links:

BibTex

@inproceedings{5248,
  title = {Semi-Supervised Laplacian Regularization of Kernel Canonical Correlation Analysis},
  journal = {Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2008},
  booktitle = {ECML PKDD 2008},
  abstract = {Kernel canonical correlation analysis (KCCA) is a dimensionality
  reduction technique for paired data. By finding directions that
  maximize correlation, KCCA learns representations that are more closely
  tied to the underlying semantics of the data rather than noise. However,
  meaningful directions are not only those that have high correlation to another
  modality, but also those that capture the manifold structure of the
  data. We propose a method that is simultaneously able to find highly
  correlated directions that are also located on high variance directions
  along the data manifold. This is achieved by the use of semi-supervised
  Laplacian regularization of KCCA. We show experimentally that Laplacian
  regularized training improves class separation over KCCA with only
  Tikhonov regularization, while causing no degradation in the correlation
  between modalities. We propose a model selection criterion based on
  the Hilbert-Schmidt norm of the semi-supervised Laplacian regularized
  cross-covariance operator, which we compute in closed form.},
  pages = {133-145},
  editors = {Daelemans, W. , B. Goethals, K. Morik},
  publisher = {Springer},
  organization = {Max-Planck-Gesellschaft},
  school = {Biologische Kybernetik},
  address = {Berlin, Germany},
  month = aug,
  year = {2008},
  slug = {5248},
  author = {Blaschko, MB. and Lampert, CH. and Gretton, A.},
  month_numeric = {8}
}