Variational Inference is a technique which consists in bounding the log-likelihood **ln p(x) **defined by a model with latent variables **p(x,z)=p(x|z)p(z)** through the introduction of a variational distribution **q(z|x)** with same support as **p(z)**:

Often the expectations in the bound **F(x) **(aka, ELBO or Free Energy) cannot be solved analytically.

In some cases, we can make use of a few handful inequalities which I quickly summarize below.

Some of these inequalities introduce new variational parameters. Those should be optimized jointly with all the other parameters to minimize the ELBO.

### Like this:

Like Loading...

Great! The piecewise linear/quadratic bound for \log (1+\exp(x)) here could also come in handy http://www.icml-2011.org/papers/376_icmlpaper.pdf

Thanks, indeed!