Electrical Engineering
      and Computer Sciences

Electrical Engineering and Computer Sciences

COLLEGE OF ENGINEERING

UC Berkeley

Real-time Musical Score Following from a Live Audio Source

Andrew Lee

EECS Department
University of California, Berkeley
Technical Report No. UCB/EECS-2014-127
May 22, 2014

http://www.eecs.berkeley.edu/Pubs/TechRpts/2014/EECS-2014-127.pdf

Musical score following has many practical applications. As mobile devices get more popular, musicians would like to carry their sheet music on their mobile devices. One useful functionality that a sheet music app could offer is to have the app follow the music being played by the musician and turn pages at the right time. In this paper, we describe how we combined several techniques to accomplish this task and how we implemented it to run efficiently. By understanding basic sheet music semantics and how instruments produce sound, we carefully constructed a hidden Markov model to follow a player's position in a musical score solely by listening to their playing. We implemented our algorithm in Java, in the form of a sheet music application for Android, and it was able to successfully follow amateur-level pianists playing Twinkle Twinkle Little Star, as well as Minuet in G, by J.S. Bach.

Advisor: Maneesh Agrawala


BibTeX citation:

@mastersthesis{Lee:EECS-2014-127,
    Author = {Lee, Andrew},
    Editor = {Agrawala, Maneesh and Séquin, Carlo H.},
    Title = {Real-time Musical Score Following from a Live Audio Source},
    School = {EECS Department, University of California, Berkeley},
    Year = {2014},
    Month = {May},
    URL = {http://www.eecs.berkeley.edu/Pubs/TechRpts/2014/EECS-2014-127.html},
    Number = {UCB/EECS-2014-127},
    Abstract = {Musical score following has many practical applications. As mobile devices get more popular, musicians would like to carry their sheet music on their mobile devices. One useful functionality that a sheet music app could offer is to have the app follow the music being played by the musician and turn pages at the right time. In this paper, we describe how we combined several techniques to accomplish this task and how we implemented it to run efficiently. By understanding basic sheet music semantics and how instruments produce sound, we carefully constructed a hidden Markov model to follow a player's position in a musical score solely by listening to their playing. We implemented our algorithm in Java, in the form of a sheet music application for Android, and it was able to successfully follow amateur-level pianists playing <i>Twinkle Twinkle Little Star</i>, as well as <i>Minuet in G</i>, by J.S. Bach.}
}

EndNote citation:

%0 Thesis
%A Lee, Andrew
%E Agrawala, Maneesh
%E Séquin, Carlo H.
%T Real-time Musical Score Following from a Live Audio Source
%I EECS Department, University of California, Berkeley
%D 2014
%8 May 22
%@ UCB/EECS-2014-127
%U http://www.eecs.berkeley.edu/Pubs/TechRpts/2014/EECS-2014-127.html
%F Lee:EECS-2014-127