2008 Research Summary

Natural Speech Interfaces for Environment and Device Control (SNAC)

View Current Project Information

John F. Canny and Ana Ramirez Chang

This project explores "natural speech" interfaces in a ubiquitous computing environment. By natural speech, we mean interfaces that depend only on users' natural forms of expression, not on a learned command language. Specifically, we explore the invocation of custom lighting patterns in an "open-plan" workplace. The workspace contains a large array of individually-dimmable downlights, which is very flexible but expensive to configure for common tasks. We argue that speech is a good solution for this task, but raises two of the generic challenges for natural human-machine interaction: (i) discovering how users give commands and (ii) discovering the configurations named by those commands. It is not just a problem of building a language model or grammar for user input, but rather discovering the semantics of those inputs. We call this problem "SNAC" for Simultaneous Naming and Configuration. We are exploring a solution for this task, based on a modified wizard-of-Oz study to elicit natural control utterances from users followed by data analysis to discover command/configuration tuples.