Electrical Engineering
      and Computer Sciences

Electrical Engineering and Computer Sciences

COLLEGE OF ENGINEERING

UC Berkeley

Beta processes, stick-breaking, and power laws

Tamara Broderick, Michael Jordan and Jim Pitman

EECS Department
University of California, Berkeley
Technical Report No. UCB/EECS-2011-125
December 8, 2011

http://www.eecs.berkeley.edu/Pubs/TechRpts/2011/EECS-2011-125.pdf

The beta-Bernoulli process provides a Bayesian nonparametric prior for models involving collections of binary-valued features. A draw from the beta process yields an infinite collection of probabilities in the unit interval, and a draw from the Bernoulli process turns these into binary-valued features. Recent work has provided stick-breaking representations for the beta process analogous to the well-known stick-breaking representation for the Dirichlet process. We derive one such stick-breaking representation directly from the characterization of the beta process as a completely random measure. This approach motivates a three-parameter generalization of the beta process, and we study the power laws that can be obtained from this generalized beta process. We present a posterior inference algorithm for the beta-Bernoulli process that exploits the stick-breaking representation, and we present experimental results for a discrete factor-analysis model.

Advisor: Michael Jordan


BibTeX citation:

@mastersthesis{Broderick:EECS-2011-125,
    Author = {Broderick, Tamara and Jordan, Michael and Pitman, Jim},
    Title = {Beta processes, stick-breaking, and power laws},
    School = {EECS Department, University of California, Berkeley},
    Year = {2011},
    Month = {Dec},
    URL = {http://www.eecs.berkeley.edu/Pubs/TechRpts/2011/EECS-2011-125.html},
    Number = {UCB/EECS-2011-125},
    Abstract = {The beta-Bernoulli process provides a Bayesian nonparametric prior for models involving collections of binary-valued features. A draw from the beta process yields an infinite collection of probabilities in the unit interval, and a draw from the Bernoulli process turns these into binary-valued features. Recent work has provided stick-breaking representations for the beta process analogous to the well-known stick-breaking representation for the Dirichlet process. We derive one such stick-breaking representation directly from the characterization of the beta process as a completely random measure. This approach motivates a three-parameter generalization of the beta process, and we study the power laws that can be obtained from this generalized beta process. We present a posterior inference algorithm for the beta-Bernoulli process that exploits the stick-breaking representation, and we present experimental results for a discrete factor-analysis model.}
}

EndNote citation:

%0 Thesis
%A Broderick, Tamara
%A Jordan, Michael
%A Pitman, Jim
%T Beta processes, stick-breaking, and power laws
%I EECS Department, University of California, Berkeley
%D 2011
%8 December 8
%@ UCB/EECS-2011-125
%U http://www.eecs.berkeley.edu/Pubs/TechRpts/2011/EECS-2011-125.html
%F Broderick:EECS-2011-125