Dutch-CAPT (NWO PROO) project

Implications of Potentially Erroneous Feedback in CALL Systems

(In Dutch: "Implicaties van potentieel foutieve feedback voor CALL-systemen")

The ultimate objective of this project is to advance our understanding of the effects on learning of the frequency and the seriousness of feedback errors in advanced Computer Assisted Language Learning (CALL) systems. The domain in which we work is pronunciation training in the acquisition of Dutch as a second language. Pronunciation training is an outstanding example of a situation in which the need for interactive multimedia environments is obvious. At the same time it is obvious that state-of-the-art speech and language technology will make occasional assessment errors when it comes to evaluating the correctness of responses in interactive student-system dialogs, both with respect to contents and pronunciation quality or fluency. Empirical investigation of the impact of feedback errors in interactive multimedia courses requires that such courses are available.

For this purpose, we have developed Computer-Assisted Pronunciation Training (CAPT) program based on Automatic Speech Recognition for learners of Dutch with different mother tongues (Dutch-CAPT). The program offers realistic language input in the form of short movies with Dutch actors and audio recordings of several native speakers. It contains a wide range of pronunciation exercises including minimal pairs and dialogues and it provides automatic, easy-to-understand, and reliable feedback on the student's pronunciation, by focussing on eleven problematic Dutch sounds. The program, whose design has been based on a thorough analysis of typical segmental errors made by non-native speakers of Dutch, has been tested in a series of experiments. The subjects of these experiments were beginner learners from various countries across the world, who were following a Dutch course. One group of subjects used this program for a few hours over a period of four weeks; another group used a program that did not provide any feedback on pronunciation for approximately the same amount of time; a third group who did not train with any program was used as control. The pronunciation of these subjects before and after training was scored along the dimension 'segmental quality' by a number of experts. The results of these experiments, which are currently being examined, will reveal whether this program, which has the advantage of providing immediate and automatic feedback, does indeed lead to significant improvements in the learner's pronunciation, and whether it is conducive to higher learning gains than more traditional forms of training without feedback.

This project has been carried out within the framework of the NWO 'Programma voor onderwijsonderzoek' (PROO).
 

Publications

 

Presentations

 

The pedagogical effectiveness of ASR-based computer assisted pronunciation training

Ambra Neri
PhD Thesis, University of Nijmegen [PDF]

Abstract:
Computer Assisted Pronunciation Training (CAPT) systems with Automatic Speech Recognition (ASR) technology have become increasingly popular to train pronunciation in the second language (L2). The advantage of these systems is the provision of a self-paced, stress-free type of training with automatic feedback on pronunciation quality. Despite this popularity, little is known on the actual pedagogical effectiveness of these systems. In other words, little empirical evidence is available as to whether and to what extent the use of these systems can improve pronunciation quality for a learner, while it is well-known that ASR-based feedback on non-native pronunciation quality is not yet 100% error-free. The research reported on in this thesis investigates the pedagogical effectiveness of ASR-based feedback on segmental quality. The thesis starts by identifying pedagogical requirements for pronunciation training in L2. Existing CAPT systems are then critically examined to establish which pedagogical requirements can be achieved with current ASR-based CAPT technology. Some of these suggestions are subsequently implemented to develop a customized ASR-based CAPT system (Dutch-CAPT) for teaching Dutch pronunciation to adult immigrants. First, a method is presented to select important segmental errors made by learners of Dutch with different mother tongues. By means of auditory analyses of different speech databases, an inventory of eight Dutch phonemes that appear to be particularly problematic tp learn is obtained. This inventory is subsequently implemented in Dutch-CAPT, which offers a simple form of feedback on segmental errors. The improvement made by a group of immigrants who used Dutch-CAPT is measured and compared to that of controls. The results indicate that the ASR-based feedback provided yielded the largest improvements in the pronunciation of the targeted phonemes, despite occasional errors in the feedback. The thesis ends with suggestions to design pedagogically sound and technologically reliable ASR-based CAPT, and to evaluate these systems.

Before defending her PhD thesis Ambra gave a (so called 'layman') presentation about her work for the people present at her defense. [PPT] [PDF] [PDF handout]
 

Links

PhD Thesis Ambra Neri:
* http://lands.let.ru.nl/~strik/publications/theses/Neri-PhD-thesis.pdf
* http://www.darenet.nl/nl/page/repository.item/show?saharaIdentifier=ru:oai:repository.ubn.ru.nl:2066/30022
* http://webdoc.ubn.ru.nl/mono/n/neri_a/pedaefofa.pdf

http://lands.let.ru.nl/staff/neri.php - publications Ambra Neri

More information about the Dutch-CAPT (NWO PROO) project:
* http://lands.let.ru.nl/projects/proo.html
* http://www.nwo.nl/projecten.nsf/pages/1500109115