<-- Go back

VarGrad: A Low-Variance Gradient Estimator for Variational Inference

by Lorenz Richter, Ayman Boustati, Nikolas Nüsken, Francisco J. R. Ruiz, Ömer Deniz Akyildiz

Year:

2020

Publication:

eprint arXiv:2010.10436

Abstract:

We analyse the properties of an unbiased gradient estimator of the ELBO for variational inference, based on the score function method with leave-one-out control variates. We show that this gradient estimator can be obtained using a new loss, defined as the variance of the log-ratio between the exact posterior and the variational approximation, which we call the log-variance loss.

Link:

Read the paper

Additional Information


Brief introduction of the dida co-author(s) and relevance for dida's ML developments.

About the Co-Author

With an original focus on stochastics and numerics (FU Berlin), the mathematician has been dealing with deep learning algorithms for some time now. Besides his interest in the theory, he has practically solved multiple data science problems in the last 10 years. Lorenz leads the machine learning team.