Empirical Inference Conference Paper 2005

Measuring Statistical Dependence with Hilbert-Schmidt Norms

We propose an independence criterion based on the eigenspectrum of covariance operators in reproducing kernel Hilbert spaces (RKHSs), consisting of an empirical estimate of the Hilbert-Schmidt norm of the cross-covariance operator (we term this a Hilbert-Schmidt Independence Criterion, or HSIC). This approach has several advantages, compared with previous kernel-based independence criteria. First, the empirical estimate is simpler than any other kernel dependence test, and requires no user-defined regularisation. Second, there is a clearly defined population quantity which the empirical estimate approaches in the large sample limit, with exponential convergence guaranteed between the two: this ensures that independence tests based on {methodname} do not suffer from slow learning rates. Finally, we show in the context of independent component analysis (ICA) that the performance of HSIC is competitive with that of previously published kernel-based criteria, and of other recently published ICA methods.

Author(s): Gretton, A. and Bousquet, O. and Smola, A. and Schoelkopf, B.
Book Title: Algorithmic Learning Theory, Lecture Notes in Computer Science, Vol. 3734
Journal: Algorithmic Learning Theory: 16th International Conference, ALT 2005
Pages: 63-78
Year: 2005
Month: October
Day: 8
Editors: S Jain and H-U Simon and E Tomita
Publisher: Springer
Bibtex Type: Conference Paper (inproceedings)
Address: Berlin, Germany
DOI: 10.1007/11564089_7
Event Name: 16th International Conference ALT 2005
Event Place: Singapore
Digital: 0
Electronic Archiving: grant_archive
Language: en
Organization: Max-Planck-Gesellschaft
School: Biologische Kybernetik
Links:

BibTex

@inproceedings{3774,
  title = {Measuring Statistical Dependence with Hilbert-Schmidt Norms},
  journal = {Algorithmic Learning Theory: 16th International Conference, ALT 2005},
  booktitle = {Algorithmic Learning Theory, Lecture Notes in Computer Science, Vol. 3734},
  abstract = {We propose an independence criterion based on the eigenspectrum of covariance operators in reproducing kernel Hilbert spaces (RKHSs), consisting of an empirical estimate of the Hilbert-Schmidt norm of the cross-covariance operator  (we term this a Hilbert-Schmidt Independence Criterion, or HSIC).  This approach has several advantages, compared with previous kernel-based independence criteria.  First, the empirical estimate is simpler than any other kernel dependence test, and requires no user-defined regularisation. Second, there is a clearly defined population quantity which the empirical estimate approaches in the large sample limit, with exponential convergence guaranteed between the two: this ensures that independence tests based on {methodname} do not suffer from slow learning rates.
  Finally, we show in the context of independent component analysis (ICA) that the performance of HSIC is competitive with that of previously published kernel-based criteria, and of other recently published ICA methods.},
  pages = {63-78},
  editors = {S Jain and H-U Simon and E Tomita},
  publisher = {Springer},
  organization = {Max-Planck-Gesellschaft},
  school = {Biologische Kybernetik},
  address = {Berlin, Germany},
  month = oct,
  year = {2005},
  slug = {3774},
  author = {Gretton, A. and Bousquet, O. and Smola, A. and Schoelkopf, B.},
  month_numeric = {10}
}