Empirical Inference Technical Report 2008

Block-Iterative Algorithms for Non-Negative Matrix Approximation

In this report we present new algorithms for non-negative matrix approximation (NMA), commonly known as the NMF problem. Our methods improve upon the well-known methods of Lee & Seung [19] for both the Frobenius norm as well the Kullback-Leibler divergence versions of the problem. For the latter problem, our results are especially interesting because it seems to have witnessed much lesser algorithmic progress as compared to the Frobenius norm NMA problem. Our algorithms are based on a particular block-iterative acceleration technique for EM, which preserves the multiplicative nature of the updates and also ensures monotonicity. Furthermore, our algorithms also naturally apply to the Bregman-divergence NMA algorithms of Dhillon and Sra [8]. Experimentally, we show that our algorithms outperform the traditional Lee/Seung approach most of the time.

Author(s): Sra, S.
Number (issue): 176
Year: 2008
Month: September
Day: 0
Bibtex Type: Technical Report (techreport)
Digital: 0
Electronic Archiving: grant_archive
Institution: Max-Planck Institute for Biological Cybernetics, Tübingen, Germany
Language: en
Organization: Max-Planck-Gesellschaft
School: Biologische Kybernetik
Links:

BibTex

@techreport{5556,
  title = {Block-Iterative Algorithms for
  Non-Negative Matrix Approximation},
  abstract = {In this report we present new algorithms for non-negative matrix approximation (NMA),
  commonly known as the NMF problem. Our methods improve upon the well-known methods of Lee &
  Seung [19] for both the Frobenius norm as well the Kullback-Leibler divergence versions of the problem.
  For the latter problem, our results are especially interesting because it seems to have witnessed much
  lesser algorithmic progress as compared to the Frobenius norm NMA problem. Our algorithms are
  based on a particular block-iterative acceleration technique for EM, which preserves the multiplicative
  nature of the updates and also ensures monotonicity. Furthermore, our algorithms also naturally apply
  to the Bregman-divergence NMA algorithms of Dhillon and Sra [8]. Experimentally, we show that our
  algorithms outperform the traditional Lee/Seung approach most of the time.},
  number = {176},
  organization = {Max-Planck-Gesellschaft},
  institution = {Max-Planck Institute for Biological Cybernetics, Tübingen, Germany},
  school = {Biologische Kybernetik},
  month = sep,
  year = {2008},
  slug = {5556},
  author = {Sra, S.},
  month_numeric = {9}
}