Deterministic annealing for semi-supervised kernel machines
An intuitive approach to utilizing unlabeled data in kernel-based classification algorithms is to simply treat the unknown labels as additional optimization variables. For margin-based loss functions, one can view this approach as attempting to learn low-density separators. However, this is a hard optimization problem to solve in typical semi-supervised settings where unlabeled data is abundant. The popular Transductive SVM algorithm is a label-switching-retraining procedure that is known to be susceptible to local minima. In this paper, we present a global optimization framework for semi-supervised Kernel machines where an easier problem is parametrically deformed to the original hard problem and minimizers are smoothly tracked. Our approach is motivated from deterministic annealing techniques and involves a sequence of convex optimization problems that are exactly and efficiently solved. We present empirical results on several synthetic and real world datasets that demonstrate the effectiveness of our approach.
Author(s): | Sindhwani, V. and Keerthi, SS. and Chapelle, O. |
Book Title: | ICML 2006 |
Journal: | Proceedings of the 23rd International Conference on Machine Learning (ICML 2006) |
Pages: | 841-848 |
Year: | 2006 |
Month: | June |
Day: | 0 |
Editors: | Cohen, W. W., A. Moore |
Publisher: | ACM Press |
Bibtex Type: | Conference Paper (inproceedings) |
Address: | New York, NY, USA |
DOI: | 10.1145/1143844.1143950 |
Event Name: | 23rd International Conference on Machine Learning |
Event Place: | Pittsburgh, PA, USA |
Digital: | 0 |
Electronic Archiving: | grant_archive |
Language: | en |
Organization: | Max-Planck-Gesellschaft |
School: | Biologische Kybernetik |
Links: |
BibTex
@inproceedings{4061, title = {Deterministic annealing for semi-supervised kernel machines}, journal = {Proceedings of the 23rd International Conference on Machine Learning (ICML 2006)}, booktitle = {ICML 2006}, abstract = {An intuitive approach to utilizing unlabeled data in kernel-based classification algorithms is to simply treat the unknown labels as additional optimization variables. For margin-based loss functions, one can view this approach as attempting to learn low-density separators. However, this is a hard optimization problem to solve in typical semi-supervised settings where unlabeled data is abundant. The popular Transductive SVM algorithm is a label-switching-retraining procedure that is known to be susceptible to local minima. In this paper, we present a global optimization framework for semi-supervised Kernel machines where an easier problem is parametrically deformed to the original hard problem and minimizers are smoothly tracked. Our approach is motivated from deterministic annealing techniques and involves a sequence of convex optimization problems that are exactly and efficiently solved. We present empirical results on several synthetic and real world datasets that demonstrate the effectiveness of our approach.}, pages = {841-848}, editors = {Cohen, W. W., A. Moore}, publisher = {ACM Press}, organization = {Max-Planck-Gesellschaft}, school = {Biologische Kybernetik}, address = {New York, NY, USA}, month = jun, year = {2006}, slug = {4061}, author = {Sindhwani, V. and Keerthi, SS. and Chapelle, O.}, month_numeric = {6} }