Language Technologies in Speech-Enabled Second Language ...

Feb 28, 2012 - of the Requirements for the Degree of Ph.D in Electrical Engineering and Computer Science. ABSTRACT. Second language learning has ...
2MB Sizes 0 Downloads 170 Views
Language Technologies in Speech-Enabled Second Language Learning Games: From Reading to Dialogue By Yushi Xu S.M. in Electrical Engineering and Computer Science (2008) Massachusetts Institute of Technology

Submitted to Department of Electrical Engineering and Computer Science in Partial Fulfillment of the Requirements for the Degree of Ph.D

at the Massachusetts Institute of Technology June, 2012

© 2012 Massachusetts Institute of Technology All rights reserved

Signature of the author ___________________________________________________________ Department of Electrical Engineering and Computer Science February 28, 2012 Certified by ____________________________________________________________________ Stephanie Seneff Senior Research Scientist Thesis Supervisor Accepted by ___________________________________________________________________ Leslie A. Kolodziejski Chair, Department Committee on Graduate Students

2

Language Technologies in Speech-Enabled Second Language Learning Games: From Reading to Dialogue By Yushi Xu

Submitted to Department of Electrical Engineering and Computer Science In February 28, 2012 in Partial Fulfillment of the Requirements for the Degree of Ph.D in Electrical Engineering and Computer Science

ABSTRACT Second language learning has become an important societal need over the past decades. Given that the number of language teachers is far below demand, computer-aided language learning software is becoming a promising supplement to traditional classroom learning, as well as potentially enabling new opportunities for self-learning. The use of speech technologies is especially attractive to offer students unlimited chances for speaking exercises. To create helpful and intelligent speaking exercises on a computer, it is necessary for the computer to not only recognize the acoustics, but also to understand the meaning and give appropriate responses. Nevertheless, most existing speech-enabled language learning software focuses only on speech recognition and pronunciation training. Very few have emphasized exercising the student’s composition and comprehension abilities and adopting language technologies to enable free-form conversation emulating a real human tutor. This thesis investigates the critical functionalities of a computer-aided language learning system, and presents a generic framework as well as various language- and domain-independent modules to enable building complex speech-based language learning systems. Four games have been designed and implemented using the framework and the modules to demonstrate their usability and flexibility, where dynamic content creation, automatic assessment, and automatic assistance are emphasized. The four games, reading, translation, question-answering and dialogue, offer different activities with gradually increasing difficulty, and involve a wide range of language processing techniques, such as language understanding, language generation, question generation, context resolution, dialogue management and user simulation. User studies with real subjects show that the systems were well received and judged to be helpful.

Thesis supervisor: Stephanie Seneff Title: Senior Research Scientist

3

4

Acknowledgments I would like to express my sincere gratitude to my supervisor Stephanie Seneff, who has given me guidance and enormous encouragements. Her earnest in research and kindness as a supervisor and a colleague strongly influence me during my Master’s and Ph.D courses, and will continue having impact on me in a longer term. I would also like to thank my committee, Regina Barzilay and Victor Zue, for giving me helpful suggestions on the thesis. This thesis would not be completed without my collaborators and colleagues. I would like to thank Scott Cyphers and Lee Hetherington for offering all kinds of hardware and software support. I would like to thank Ian McGraw for his generous help on WAMI. I would like to thank An