Research Projects

Unsupervised Learning of Visual Sense Models for Polysemous Words

Trevor Darrell and Kate Saenko¹

Polysemy is a problem for methods that exploit image search engines to build object category models. Existing unsupervised approaches do not take word sense into consideration. We propose a new method that uses a dictionary to learn models of visual word sense from a large collection of unlabeled web data. The use of LDA to discover a latent sense space makes the model robust despite the very limited nature of dictionary definitions. The definitions are used to learn a distribution in the latent space that best represents a sense. The algorithm then uses the text surrounding image links to retrieve images with high probability of a particular dictionary sense. An object classifier is trained on the resulting sense-specific images. We have shown that our method obtains higher accuracy than a method based on generating sense-specific search terms.

Figure 1: Model Overview

[1]

Kate Saenko and Trevor Darrell, Unsupervised Learning of Visual Sense Models for Polysemous Words, NIPS 2008

[2]

Kate Saenko and Trevor Darrell,Filtering Abstract Senses From Image Search Results, NIPS 2009

¹MIT