Psycholinguistic Approach to Korean/English Vocabulary Acquisition using Speech Recognition Technology within Mobile Applications

One of the most common challenges for both KFL (Korean as a foreign language) and ESL learners has related to the acquisition of new vocabulary including the pronunciation, and intonation of target utterances. However, no mobile app currently meets such needs of these language learners. To address this issue, Dr. Yong-Taek Kim (PI), Dr. Karen Head (Co-PI), and Dr. Rob Griffin (ESL/ELL Specialist in the CommLab) will develop a mobile application for language acquisition that cohesively integrates recent developments in evidence-based learning, biofeedback techniques, and speech recognition. The principal investigators will be assisted by Ethan Roland, a junior student in Computer Science and Huijin Chung, a junior student in Industrial Engineering.

Both KFL and ESL learners will raise phonemic and phonological awareness of Korean and English by comparing the sound waves of target Korean/English utterances by a native speaker with their own output via speech recognition and phonemic/prosody feedback technology. Also, it will provide either a photo or an animated image of a Korean or English word and give options for students to select the appropriate Korean or English word. The app will not include corresponding English or Korean vocabulary but offer instead other hints, such as pronunciation prompts, photos for nouns or adjectives, and animated images for verbs or adverbs, in order to minimize the interference of the students' mother tongue, English or Korean.

Another crucial feature of this app is the spaced-repetition of tasks, an evidence-based learning technique that is usually performed with flashcards. Newly introduced and more difficult flashcards are shown more frequently while older and easier flashcards are shown less frequently to exploit the psychological spacing effect. The use of spaced repetition has been shown to increase the rate of learning and retention.

There will be several potential usages for this application, including the instruction of KFL/ESL in the classroom or phonology lab, or, as a language application for learners on their personal devices for individual vocabulary and pronunciation practice at home.

Project Year
Project Leads
Yongtaek Kim
Robert Griffin
Karen Head
Ethan Roland
Huijin Chung
Contact Email