|
|
"I stand at the window and see a house, trees, sky. Theoretically I might say
there were 327 brightnesses and nuances of colour. Do I have "327"? No. I have
sky, house, and trees." --Max Wertheimer
|
Overview:
The phenomenon of visual grouping was first highlighted by the Gestalt
school of visual perception led by Max Wertheimer, nearly a century ago.
In computational vision, this ability has been studied as "image
segmentation", the partitioning of an image (or video stream) into sets
of pixels that correspond to "objects" or parts of objects. This
process is based on bottom up cues such as similarity of pixel
brightness, color, texture and motion as well as top down input derived
from familiar object categories such as faces. Our research is aimed at
developing a scientific understanding of grouping, both in the context
of human perception and for computer vision. Key contributions include:
- A large dataset of natural images that have been segmented by human
observers. This dataset, available [here], serves
as ground truth for learning grouping cues as well as a benchmark for comparing
different segmentation and boundary finding algorithms.
- Computational models of low level cues such as brightness, color,
texture and motion, inspired by corresponding mechanisms in primary
visual cortex. We use the dataset of human segmented images to calibrate
the cues and learn optimum cue combination strategies. The models thus
reflect the "ecological statistics" of natural images, which presumably
visual animals have evolved to exploit.
- Algorithms that go from local image cues to globally optimal
segmentations. The normalized cut algorithm, based on spectral graph
theory, provides a mechanism for going from pairwise pixel affinities
based on local cues to partitions that maximize the ratio of affinities
within a group to that across groups. By computing oriented gradients on the
eigenvectors of the normalized Laplacian, we get a robust signal for marking
image contours. We have used Conditional Random Fields as a formalism for
combing low-, mid- and high- level cues for grouping and figure-ground discrimination.
Publications:
The Berkeley Segmentation Data Set is available [here].
|
"From Contours to Regions: An Empirical Evaluation"
P. Arbelaez,
M. Maire,
C. Fowlkes, and
J. Malik.
CVPR 2009
[pdf]
Code available:
[64-bit Linux MEX Binaries and Source]
|
|
"Using Contours to Detect and Localize Junctions in Natural Images"
M. Maire,
P. Arbelaez,
C. Fowlkes, and
J. Malik.
CVPR 2008
[pdf]
[slides]
Code for Global Pb (gPb):
[Full package with Matlab 64-bit Linux MEX files]
[Source code for MEX files] (Distributed under the GNU AGPL)
Note: The precompiled package also includes the segment program from the BSE.
|
|
"Learning Probabilistic Models for Contour Completion in Natural Images"
X. Ren,
C. Fowlkes, and
J. Malik.
IJCV 2008
[pdf]
|
|
"Local Figure/Ground Cues are Valid for Natural Images"
C. Fowlkes,
D. Martin, and
J. Malik.
Journal of Vision 2007
[pdf]
|
|
|
"Figure/Ground Assignment in Natural Images"
X. Ren,
C. Fowlkes, and
J. Malik.
ECCV 2006
[pdf]
|
|
"Cue Integration in Figure/Ground Labeling"
X. Ren,
C. Fowlkes, and
J. Malik.
NIPS 2005
[pdf]
|
|
|
"Scale-Invariant Contour Completion using Conditional Random Fields"
X. Ren,
C. Fowlkes, and
J. Malik.
ICCV 2005
[pdf]
[ps.gz]
|
|
|
"Mid-level Cues Improve Boundary Detection"
X. Ren,
C. Fowlkes, and
J. Malik.
Technical Report CSD-05-1382
[pdf]
|
|
"How Much Does Globalization Help Segmentation?"
C. Fowlkes, and
J. Malik.
Technical Report CSD-04-1340
[pdf]
|
|
"Learning a Classification Model for Segmentation"
X. Ren, and
J. Malik.
ICCV 2003
[pdf]
[ps.gz]
|
|
"Spectral Grouping Using the Nyström Method"
C. Fowlkes,
S. Belongie,
F. Chung and
J. Malik.
IEEE PAMI, Feb. 2004
[pdf]
|
|
"Learning to Detect Natural Image Boundaries Using Local Brightness, Color, and Texture Cues"
D. Martin,
C. Fowlkes, and
J. Malik.
to appear in PAMI
[pdf]
|
|
"Learning Affinity Functions for Image Segmentation: Combining
Patch-based and Gradient-based Approaches"
C. Fowlkes,
D. Martin, and
J. Malik.
CVPR 2003
[pdf] [ps.gz]
|
|
|
"Learning to Detect Natural Image Boundaries Using Brightness and Texture"
D. Martin,
C. Fowlkes, and
J. Malik.
NIPS 2002
[pdf]
[ps.gz]
|
|
"A Probabilistic Multi-scale Model for Contour Completion Based on Image Statistics"
X. Ren,
J. Malik.
ECCV 2002
[pdf]
|
|
|
"Spectral Partitioning with Indefinite Kernels using the Nyström Extension"
S. Belongie,
C. Fowlkes,
F. Chung and
J. Malik.
ECCV 2002
[pdf]
[ps.gz]
|
|
|
"Efficient Spatiotemporal Grouping Using the Nyström Method"
C. Fowlkes,
S. Belongie, and
J. Malik.
CVPR 2001
[pdf]
[ps.gz]
|
|
"A Database of Human Segmented Natural Images and its Application
to Evaluating Segmentation Algorithms and Measuring Ecological
Statistics"
D. Martin,
C. Fowlkes,
D. Tal, and
J. Malik.
ICCV 2001
[pdf]
[ps.gz]
|
|
"Contour and Texture Analysis for Image Segmentation"
J. Malik,
S. Belongie,
T. Leung and
J. Shi.
IJCV 2001
[ps.gz]
|
|
"Normalized Cuts and Image Segmentation"
J. Shi and
J. Malik.
IEEE TPAMI, 22(8) Aug. 2000
[ps.gz]
|
|
|
"Textons, Contours and Regions: Cue Combination in Image Segmentation"
J. Malik,
S. Belongie,
J. Shi and
T. Leung.
ICCV 1999
[ps.gz]
|
|
"Motion Segmentation and Tracking Using Normalized Cuts"
J. Shi and
J. Malik.
ICCV 1998
[ps.gz]
|
|
"Contour continuity in region based image segmentation"
T. Leung and
J. Malik.
ECCV 1998
[ps.gz]
|
|
"Finding Boundaries in Natural Images: A New Method Using Point Desdcriptors
and Area Completion"
S. Belongie and
J. Malik.
ECCV 1998
[ps.gz]
|
|
"Self Inducing Relation Distance and its Application to Image Segmentation"
J. Shi and
J. Malik.
ECCV 1998
[ps.gz]
|
|
|
"Normalized Cuts and Image Segmentation"
J. Shi and
J. Malik.
CVPR 1997
[ps.gz]
|
Back to Berkeley Computer Vision page
Questions --> Michael Maire
|