- Recherche et sélection de publications
|
Stochastic Quasi-Newton Langevin Monte Carlo
- Umut Simsekli #1, Roland Badeau #1, Gaël Richard #1, Ali Taylan Cemgil #2
-
| #1 |
Laboratoire Traitement et Communication de l'Information [Paris] (LTCI)
- Télécom ParisTech
- CNRS : UMR5141
|
| #2 |
Department of Computer Engineering [Bogazici]
|
- References
- ICML, New York, NY, USA, June 2016,
- Abstract
In the past few years, Stochastic Gradient Markov Chain Monte Carlo (SG-MCMC) methods have been proposed to scale up MCMC to the large-scale scheme. Whilst these methods have proven useful in many applications, they might suffer from poor mixing rates when the target densities exhibit high correlations or large scale differences. In this study, we propose a novel SG-MCMC method that takes the local correlations and scale into account by using ideas from Quasi-Newton optimization methods that directly approximate the inverse Hessian by using a limited history of samples and their gradients. Our method is both computationally efficient and powerful since it uses dense approximations of the inverse Hessian while keeping the time and memory complexities linear with the dimension of the problem. We provide formal theoretical analysis where we show that the proposed method is asymptotically unbiased and consistent with the posterior expectations. We illustrate the effectiveness of the approach on both synthetic and real datasets. Our experiments on two challenging applications (speech enhancement and distributed link prediction) show that our method achieves fast convergence rates similar to Riemannian approaches while at the same time having low computational requirements similar to diagonal preconditioning-based approaches.
- Keywords
- Category
- Paper in proceedings
- Research Area(s)
- Engineering Sciences/Signal and Image processing
- Identifier(s)
-
Bibliographic key US:ICML-16
- File(s)
-
- Export
-
- Last update
- on september 21, 2016
|
