Project Summary

Skynet uses automatic speech recognition, natural language processing, an artificially intelligent conversation agent, and speech synthesis to help learners develop conversational skills in a second language. Learners are presented with a computer character with whom they can converse in natural language and who analyzes their speech for common linguistics errors. Using Vygotsky's concept of the zone of proximal development and the complementary idea of scaffolded learning, this project gradually expands the conversational complexity to match the speaker's skill. It progressively encourages the learner to try more ambitious sentence structures and provides meta-linguistic feedback when common errors become apparent.

Skynet employs CMU Sphinx for speech recognition, University of Edinburgh's Festival for speech synthesis, and RebeccaAIML for conversational discourse, as well as a custom NLP and linguistic analysis component for structured feedback. If you would like to read more, you may download this recent report on the project.

We are currently working with a Bay Area high school to tailor our design to support the needs of immigrant students and engage the learners at their level most effectively. We hope to evaluate the effectiveness of the system with students who are just beginning to learn English as a second language.

Downloads

• Report PDF, "Skynet: Speech-enabled Conversational Agent to Support Second Language Acquisition"
• MS Windows prototype binary distribution (unzip and run "Sarah.bat")