VarGrad: A Low-Variance Gradient Estimator for Variational Inference
von Lorenz Richter, Ayman Boustati, Nikolas Nüsken, Francisco J. R. Ruiz, Ömer Deniz Akyildiz
Jahr:
2020
Publikation:
eprint arXiv:2010.10436
Abstrakt:
We analyse the properties of an unbiased gradient estimator of the ELBO for variational inference, based on the score function method with leave-one-out control variates. We show that this gradient estimator can be obtained using a new loss, defined as the variance of the log-ratio between the exact posterior and the variational approximation, which we call the log-variance loss.
Link:
Read the paperAdditional Information
Brief introduction of the dida co-author(s) and relevance for dida's ML developments.
About the Co-Author
With an original focus on stochastics and numerics (FU Berlin), the mathematician has been dealing with deep learning algorithms for some time now. Besides his interest in the theory, he has practically solved multiple data science problems in the last 10 years. Lorenz leads the machine learning team.