Poster
An Instability in Variational Inference for Topic Models
Behrooz Ghorbani · Hamidreza Hakim Javadi · Andrea Montanari
Pacific Ballroom #236
Keywords: [ Information Theory and Estimation ] [ Generative Models ] [ Bayesian Methods ] [ Approximate Inference ]
Naive mean field variational methods are the state of-the-art approach to inference in topic modeling. We show that these methods suffer from an instability that can produce misleading conclusions. Namely, for certain regimes of the model parameters, variational inference outputs a non-trivial decomposition into topics. However -for the same parameter values- the data contain no actual information about the true topic decomposition, and the output of the algorithm is uncorrelated with it. In particular, the estimated posterior mean is wrong, and estimated credible regions do not achieve the nominal coverage. We discuss how this instability is remedied by more accurate mean field approximations.
Live content is unavailable. Log in and register to view live content