Style-aware Mid-level Representation for Discovering
Visual Connections in Space and Time




People

Yong Jae Lee
Alexei A. Efros
Martial Hebert


Abstract

We present a weakly-supervised visual data mining approach that discovers connections between recurring midlevel visual elements in historic (temporal) and geographic (spatial) image collections, and attempts to capture the underlying visual style.  In contrast to existing discovery methods that mine for patterns that remain visually consistent throughout the dataset, our goal is to discover visual elements whose appearance changes due to change in time or location; i.e., exhibit consistent stylistic variations across the label space (date or geo-location).  To discover these elements, we first identify groups of patches that are stylesensitive.  We then incrementally build correspondences to find the same element across the entire dataset.  Finally, we train style-aware regressors that model each element’s range of stylistic differences.  We apply our approach to date and geo-location prediction and show substantial improvement over several baselines that do not model visual style.  We also demonstrate the method’s effectiveness on the related task of fine-grained classification.


Paper & Presentation




ICCV Paper (pdf)

Supplemental material (pdf)
Talk slides (pptx)
Video Presentation


Citation
Y. J. Lee, A. A. Efros, and M. Hebert.  Style-aware Mid-level Representation for Discovering Visual Connections in Space and Time.  Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2013.


Code

Code can be downloaded here.


Data

Our historical car dataset (CarDb as referenced in the paper) can be downloaded here.


Acknowledgements

This work was supported in part by Google, ONR MURI N000141010934, and the Intelligence Advanced Research Projects Activity (IARPA) via Air Force Research Laboratory.  The U.S. Government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright annotation thereon.  Disclaimer: The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of IARPA, AFRL or the U.S. Government.