Empirische Inferenz Book Chapter 2011

Projected Newton-type methods in machine learning

We consider projected Newton-type methods for solving large-scale optimization problems arising in machine learning and related fields. We first introduce an algorithmic framework for projected Newton-type methods by reviewing a canonical projected (quasi-)Newton method. This method, while conceptually pleasing, has a high computation cost per iteration. Thus, we discuss two variants that are more scalable, namely, two-metric projection and inexact projection methods. Finally, we show how to apply the Newton-type framework to handle non-smooth objectives. Examples are provided throughout the chapter to illustrate machine learning applications of our framework.

Author(s): Schmidt, M. and Kim, D. and Sra, S.
Book Title: Optimization for Machine Learning
Pages: 305-330
Year: 2011
Month: December
Day: 0
Editors: Sra, S., Nowozin, S. and Wright, S. J.
Publisher: MIT Press
Bibtex Type: Book Chapter (inbook)
Address: Cambridge, MA, USA
Electronic Archiving: grant_archive
ISBN: 978-0-262-01646-9
Language: en
Organization: Max-Planck-Gesellschaft
School: Biologische Kybernetik
Links:

BibTex

@inbook{6824,
  title = {Projected Newton-type methods in machine learning},
  booktitle = {Optimization for Machine Learning},
  abstract = {We consider projected Newton-type methods for solving large-scale optimization problems arising in machine learning and related fields. We first introduce an algorithmic framework for projected Newton-type methods by reviewing a canonical projected (quasi-)Newton method. This method, while conceptually pleasing, has a high computation cost per iteration. Thus, we discuss two variants that are more scalable, namely, two-metric projection and inexact
  projection methods. Finally, we show how to apply the Newton-type framework to handle non-smooth objectives. Examples are provided throughout the chapter to illustrate machine learning applications of our framework.},
  pages = {305-330},
  editors = {Sra, S., Nowozin, S. and Wright, S. J.},
  publisher = {MIT Press},
  organization = {Max-Planck-Gesellschaft},
  school = {Biologische Kybernetik},
  address = {Cambridge, MA, USA},
  month = dec,
  year = {2011},
  slug = {6824},
  author = {Schmidt, M. and Kim, D. and Sra, S.},
  month_numeric = {12}
}