logo_UCB                        logo_UPC

Publications

Pablo Arbelaez*, Jordi Pont-Tuset*, Jonathan T. Barron, Ferran Marques, Jitendra Malik
Multiscale Combinatorial Grouping
Computer Vision and Pattern Recognition (CVPR), 2014
[PDF] [BibTeX]

@inproceedings{APBMM2014,
author = {Arbel\'{a}ez, P. and Pont-Tuset, J. and Barron, J. and Marques, F. and Malik, J.},
title = {Multiscale Combinatorial Grouping},
booktitle = {Computer Vision and Pattern Recognition},
year = {2014}
}

Abstract

We propose a unified approach for bottom-up hierarchical image segmentation and object candidate generation for recognition, called Multiscale Combinatorial Grouping (MCG). For this purpose, we first develop a fast normalized cuts algorithm. We then propose a high-performance hierarchical segmenter that makes effective use of multiscale information. Finally, we propose a grouping strategy that combines our multiscale regions into highly-accurate object candidates by exploring efficiently their combinatorial space. We conduct extensive experiments on both the BSDS500 and on the PASCAL 2012 segmentation datasets, showing that MCG produces state-of-the-art contours, hierarchical regions, and object candidates. We also propose a faster version of MCG called Single-scale Combinatorial Grouping (SCG) that takes around 3 seconds to compute the hierarchy and 2 more second to compute the candidates.

Results

Hierarchical segmentation: We obtain the best results to date on BSDS500 with a boundary detection F measure of 0.75

 

Object Candidates: We obtain the best results to date on the PASCAL2012 segmentation task. We propose a new benchmarking environment that takes into account result both at class level and instance level.

 

Timing: The following table shows the computation time in seconds of MCG and SCG on the PASCAL2012 segmentation set:

 

Code

MCG benchmark: Tools to benchmark algorithms that generate segmented object candidates. We propose two different measures (jaccard index at instance and class levels) which we sweep against the number of proposed candidates. The code is in Matlab with some parts in C++ pre-compiled for Linux, Windows, and Mac.
MCG pre-trained: Code to compute MCG candidates and hierarchies (UCMs) with models pre-trained on the BSDS500 and the PASCAL 2012 segmentation datasets (im2mcg and im2ucm functions). It includes the MCG benchmark code as well. The code is in Matlab with some parts in C++ pre-compiled for Linux and Mac.
MCG full: Full code to re-train MCG (Pareto training, random forest ranking, etc.) on new datasets and on different object categories. It also includes MCG benchmark and pre-trained models. The pre-computed hierarchies at multiple scales can be found here [2Gb] but can be re-computed. The code is in Matlab with some parts in C++ pre-compiled for Linux and Mac.
DNCuts: Stand-alone Matlab code for fast eigenvector computation in Normalized Cuts segmentation.



GitHub repository with latest updates on the code.


Datasets


Object Candidates: Pre-computed object candidates (around 5000 sorted masks per image) for PASCAL segmentation 2012 (train+val) and BSDS500 (train+val+test). We provide results for the two different versions of our system, MCG and SCG:
MCG candidates
Pascal 2012 [2.5 GB]
2913 images
SCG candidates
Pascal 2012 [616 MB]
2913 images
MCG candidates
BSDS500 [360 MB]
500 images
SCG candidates
BSDS500 [89 MB]
500 images
Hierarchies (UCMs): Pre-computed segmentation hierarchies (Ultrametric Contour Maps) for PASCAL segmentation 2012 (train+val) and BSDS500 (train+val+test). We provide results for the two different versions of our system, MCG and SCG:
MCG UCMs
Pascal 2012 [813 MB]
2913 images
SCG UCMs
Pascal 2012 [780 MB]
2913 images
MCG UCMs
BSDS500 [123 MB]
500 images
SCG UCMs
BSDS500 [118 MB]
500 images