# An Introduction to Variational Methods for Graphical Models

### Michael I. Jordan, Zoubin Ghahramani, Tommi S. Jaakkola and Lawrence K. Saul

###
EECS Department

University of California, Berkeley

Technical Report No. UCB/CSD-98-980

January 1998

### http://www.eecs.berkeley.edu/Pubs/TechRpts/1998/CSD-98-980.pdf

This paper presents a tutorial introduction to the use of variational methods for inference and learning in graphical models. We present a number of examples of graphical models, including the QMR-DT database, the sigmoid belief network, the Boltzmann machine, and several variants of hidden Markov models, in which it is infeasible to run exact inference algorithms. We then introduce variational methods, showing how upper and lower bounds can be found for local probabilities, and discussing methods for extending these bounds to bounds on global probabilities of interest. Finally we return to the examples and demonstrate how variational algorithms can be formulated in each case.

BibTeX citation:

@techreport{Jordan:CSD-98-980, Author = {Jordan, Michael I. and Ghahramani, Zoubin and Jaakkola, Tommi S. and Saul, Lawrence K.}, Title = {An Introduction to Variational Methods for Graphical Models}, Institution = {EECS Department, University of California, Berkeley}, Year = {1998}, Month = {Jan}, URL = {http://www.eecs.berkeley.edu/Pubs/TechRpts/1998/5425.html}, Number = {UCB/CSD-98-980}, Abstract = {This paper presents a tutorial introduction to the use of variational methods for inference and learning in graphical models. We present a number of examples of graphical models, including the QMR-DT database, the sigmoid belief network, the Boltzmann machine, and several variants of hidden Markov models, in which it is infeasible to run exact inference algorithms. We then introduce variational methods, showing how upper and lower bounds can be found for local probabilities, and discussing methods for extending these bounds to bounds on global probabilities of interest. Finally we return to the examples and demonstrate how variational algorithms can be formulated in each case.} }

EndNote citation:

%0 Report %A Jordan, Michael I. %A Ghahramani, Zoubin %A Jaakkola, Tommi S. %A Saul, Lawrence K. %T An Introduction to Variational Methods for Graphical Models %I EECS Department, University of California, Berkeley %D 1998 %@ UCB/CSD-98-980 %U http://www.eecs.berkeley.edu/Pubs/TechRpts/1998/5425.html %F Jordan:CSD-98-980