Empirical Inference Conference Paper 2010

Semi-supervised Learning via Generalized Maximum Entropy

Various supervised inference methods can be analyzed as convex duals of the generalized maximum entropy (MaxEnt) framework. Generalized MaxEnt aims to find a distribution that maximizes an entropy function while respecting prior information represented as potential functions in miscellaneous forms of constraints and/or penalties. We extend this framework to semi-supervised learning by incorporating unlabeled data via modifications to these potential functions reflecting structural assumptions on the data geometry. The proposed approach leads to a family of discriminative semi-supervised algorithms, that are convex, scalable, inherently multi-class, easy to implement, and that can be kernelized naturally. Experimental evaluation of special cases shows the competitiveness of our methodology.

Author(s): Erkan, AN. and Altun, Y.
Book Title: JMLR Workshop and Conference Proceedings Volume 9: AISTATS 2010
Journal: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics (AISTATS 2010)
Pages: 209-216
Year: 2010
Month: May
Day: 0
Editors: Teh, Y.W. , M. Titterington
Publisher: JMLR
Bibtex Type: Conference Paper (inproceedings)
Address: Cambridge, MA, USA
Event Name: Thirteenth International Conference on Artificial Intelligence and Statistics
Event Place: Chia Laguna Resort, Italy
Digital: 0
Electronic Archiving: grant_archive
Language: en
Organization: Max-Planck-Gesellschaft
School: Biologische Kybernetik
Links:

BibTex

@inproceedings{6622,
  title = {Semi-supervised Learning via Generalized Maximum Entropy},
  journal = {Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics (AISTATS 2010)},
  booktitle = {JMLR Workshop and Conference Proceedings Volume 9: AISTATS 2010},
  abstract = {Various supervised inference methods can
  be analyzed as convex duals of the generalized
  maximum entropy (MaxEnt) framework.
  Generalized MaxEnt aims to find a
  distribution that maximizes an entropy function
  while respecting prior information represented
  as potential functions in miscellaneous
  forms of constraints and/or penalties.
  We extend this framework to semi-supervised
  learning by incorporating unlabeled data via
  modifications to these potential functions reflecting
  structural assumptions on the data
  geometry. The proposed approach leads to a
  family of discriminative semi-supervised algorithms,
  that are convex, scalable, inherently
  multi-class, easy to implement, and
  that can be kernelized naturally. Experimental
  evaluation of special cases shows the competitiveness
  of our methodology.},
  pages = {209-216},
  editors = {Teh, Y.W. , M. Titterington},
  publisher = {JMLR},
  organization = {Max-Planck-Gesellschaft},
  school = {Biologische Kybernetik},
  address = {Cambridge, MA, USA},
  month = may,
  year = {2010},
  slug = {6622},
  author = {Erkan, AN. and Altun, Y.},
  month_numeric = {5}
}