Electrical Engineering
      and Computer Sciences

Electrical Engineering and Computer Sciences

COLLEGE OF ENGINEERING

UC Berkeley

   

2009 Research Summary

Agreement-based Learning

View Current Project Information

Percy Shuo Liang, Daniel Klein and Michael Jordan

Many problems in natural language, vision, and computational biology require the joint modeling of many dependent variables. Such models often include hidden variables, which play important roles in unsupervised learning and general missing data problems. We focus on models in which the hidden variables have natural problem domain interpretations and might be the object of inference. Many models are intractable and require approximate inference.

In contrast, our approach is to train a set of tractable component models by encouraging them to agree on the hidden variables [1]. This allows us to capture non-decomposable aspects of the data while still maintaining tractability. We exhibit an objective function for our approach, derive EM-style algorithms for parameter estimation, and demonstrate their effectiveness on three challenging real-world learning tasks: grammar induction, word alignment [2], and phylogenetic hidden Markov modeling.

[1]
P. Liang, D. Klein, and M. Jordan, "Agreement-based Learning," Neural Information Processing Systems (NIPS), 2008.
[2]
P. Liang, B. Taskar, and D. Klein, "Alignment by Agreement," Proceedings of Human Language Technology and North American Association for Computational Linguistics (HLT/NAACL), 2006.