PAC-Bayesian Generalization Bound for Density Estimation with Application to Co-clustering
We derive a PAC-Bayesian generalization bound for density estimation. Similar to the PAC-Bayesian generalization bound for classification, the result has the appealingly simple form of a tradeoff between empirical performance and the KL-divergence of the posterior from the prior. Moreover, the PAC-Bayesian generalization bound for classification can be derived as a special case of the bound for density estimation. To illustrate a possible application of our bound we derive a generalization bound for co-clustering. The bound provides a criterion to evaluate the ability of co-clustering to predict new co-occurrences, thus introducing a supervised flavor to this traditionally unsupervised task.
| Author(s): | Seldin, Y. and Tishby, N. |
| Book Title: | JMLR Workshop and Conference Proceedings Volume 5: AISTATS 2009 |
| Journal: | In the proceedings of the 12th International Conference on Artificial Intelligence and Statistics (AISTATS 2009) |
| Pages: | 472-479 |
| Year: | 2009 |
| Month: | April |
| Day: | 0 |
| Publisher: | MIT Press |
| Bibtex Type: | Conference Paper (inproceedings) |
| Address: | Cambridge, MA, USA |
| Event Name: | 12th International Conference on Artificial Intelligence and Statistics |
| Event Place: | Clearwater Beach, FL, USA |
| Digital: | 0 |
| Electronic Archiving: | grant_archive |
| Language: | en |
| Organization: | Max-Planck-Gesellschaft |
| School: | Biologische Kybernetik |
| Links: | |
BibTex
@inproceedings{6592,
title = {PAC-Bayesian Generalization Bound for Density Estimation with Application to Co-clustering},
journal = {In the proceedings of the 12th International Conference on Artificial Intelligence and Statistics (AISTATS 2009)},
booktitle = {JMLR Workshop and Conference Proceedings Volume 5: AISTATS 2009},
abstract = {We derive a PAC-Bayesian generalization bound for density estimation. Similar to the PAC-Bayesian generalization bound for classification, the result has the appealingly simple form of a tradeoff between empirical performance and the KL-divergence of the posterior from the prior. Moreover, the PAC-Bayesian generalization bound for classification can be derived as a special case of the bound for density estimation. To illustrate a possible application of our bound we derive a generalization bound for co-clustering. The bound provides a criterion to evaluate the ability of co-clustering to predict new co-occurrences, thus introducing a supervised flavor to this traditionally unsupervised task.},
pages = {472-479},
publisher = {MIT Press},
organization = {Max-Planck-Gesellschaft},
school = {Biologische Kybernetik},
address = {Cambridge, MA, USA},
month = apr,
year = {2009},
slug = {6592},
author = {Seldin, Y. and Tishby, N.},
month_numeric = {4}
}