[Lecture slides]

References

Discrete State HMMs:

• A. W. Moore, Hidden Markov Models. Slides from a tutorial presentation.
• L. R. Rabiner (1989), A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Classic reference, with clear descriptions of inference and learning algorithms.

Continuous State HMMs:

• S. Roweis, Z. Ghahramani (1999), A Unifying Review of Linear Gaussian Models. Describes the Kalman filter and other Gaussian models.
• The Kalman Filter. Wikipedia article describing linear Kalman filtering, as well as nonlinear extensions.
• M. S. Arulampalam et. al. (2002), A Tutorial on Particle Filters for Online Nonlinear/Non-Gaussian Bayesian Tracking. Accessible particle filter tutorial with pseudocode for several variants.
• A. Doucet et. al. (2001), Sequential Monte Carlo Methods in Practice. This edited volume nicely surveys the particle filtering literature.

Graphical Models:

• K. Murphy, A Brief Introduction to Graphical Models and Bayesian Networks. Online graphical model tutorial, with references.
• M. I. Jordan (2004), Graphical Models. Tutorial introduction to graphical models, inference, and learning.
• C. M. Bishop (2006), Pattern Recognition and Machine Learning. Recent textbook with a nice chapter on graphical models.

Applications and Demonstrations:

• S. Thrun, W. Burgard, D. Fox (2005), Probabilistic Robotics. Describes applications of HMMs and particle filters in robotics, including result videos.
• The Condensation Algorithm. Visual tracking using particle filters.
• Sphinx-4. An open source speech recognition system which employs HMMs.
• E. Birney (2001), Hidden Markov Models in Biological Sequence Analysis. Applications in bioinformatics.

Software:

• Kevin Murphy's Matlab toolboxes: Hidden Markov models, Kalman filters, and Bayesian networks (directed graphical models).
• Recursive Bayesian Estimation Library (ReBEL): Matlab code for linear/extended/unscented Kalman filters, and particle filters.
• R packages: sspir (linear state space models, Kalman filters), hmm.discnp (discrete-state HMMs with discrete observations), HiddenMarkov (discrete-state HMMs with continuous observations), and RHmm (discrete-state HMMs, but seems buggy and unstable).
• Hidden Markov Model Toolkit (HTK): HMM code distributed as C libraries, focused on speech recognition.
• General Hidden Markov Model (GHMM) library: Another HMM package written in C.
• BUGS: Monte Carlo methods for Bayesian inference in graphical models, including the WinBUGS graphical inteface.
• libDAI: C++ library for variational approximate inference in graphical models with discrete variables.
• Intel Probabilistic Network Library: Eventually intended to support a wide range of graphical model types and inference algorithms, but the current beta release is more limited.