contact
home > research > ATOP >

Automatic Testing of Oral Proficiency (ATOP)

Automatic Testing of Oral Proficiency (ATOP)

Every year in the Netherlands many learners of Dutch as a second language take part in examinations in which human raters evaluate their proficiency in speaking Dutch. In order to achieve greater efficiency and lower costs, attempts are being made to automate at least part of the testing procedure. Recent advances in speech recognition research suggest that there are possibilities of using computers to test oral proficiency (Townshend et al., 1998).

The aim of this project is to develop a system for automatic assessment of foreign speakers' oral proficiency in Dutch by using automatic speech recognition technology. In this project (a Senter project) the University of Nijmegen cooperates with the Dutch National Institute for Educational Measurement (CITO), Swets Test Services of Swets & Zeitlinger and PTT Telecom.

The results of the project show that automatic testing of oral proficiency is possible, and the scores obtained correlate well with human judgements. Especially oral fluency can be assessed well by means of automatically calculated temporal measures.

Duaration: 1996-1999.

Publications

C. Cucchiarini, H. Strik, L. Boves (2002)
Quantitative assessment of second language learners' fluency: Comparisons between read and spontaneous speech.
J. of the Acoustical Society of America, Vol. 111 (6), pp. 2862-2873. [PDF]
C. Cucchiarini, H. Strik, D. Binnenpoorte & L. Boves (2002)
Pronunciation Evaluation in Read and Spontaneous Speech: A Comparison between Human Ratings and Automatic Scores.
Proc. of the Fourth Int. Symposium on the Acquisition of Second-Language Speech (New Sounds 2000), University of Klagenfurt, 2002, pp. 72-79. [PDF]
H. Strik, C. Cucchiarini, D. Binnenpoorte (2000)
L2 Pronunciation Quality in Read and Spontaneous Speech.
Proc. of ICSLP-2000, the 6th International Conference on Spoken Language Processing, Beijing, China, 16-20 October 2000, pp. 582-585. [PDF]
C. Cucchiarini, H. Strik, D. Binnenpoorte & L. Boves (2000)
Towards an Automatic Oral Proficiency Test for Dutch as a Second Language: Automatic Pronunciation Assessment in Read and Spontaneous Speech.
Proc. of the InSTIL 2000 Symposium. University of Abertay Dundee, 29-30 August 2000, pp. 18-25. [PDF]
C. Cucchiarini, H. Strik, L. Boves (2000)
Different aspects of expert pronunciation quality ratings and their relation to scores produced by speech recognition algorithms.
Speech Communication, Vol. 30 (2-3), pp. 109-119. [PDF] [HTML]
C. Cucchiarini, H. Strik, L. Boves (2000)
Quantitative assessment of second language learners' fluency by means of automatic speech recognition technology.
J. of the Acoustical Society of America, Vol. 107 (2), pp. 989-999. [PDF] [HTML]
H. Strik & C. Cucchiarini (1999)
Automatic assessment of second language learners' fluency.
Proc. of the 14th International Congress of Phonetic Sciences (ICPhS-99), San Francisco, USA, 1999, pp. 759-762. [PDF] [PS]
C. Cucchiarini, H. Strik, L. Boves (1998)
Quantitative assessment of second language learners' fluency: an automatic approach.
Proc. of the fifth Int. Conference on Spoken Language Processing (ICSLP'98), 30 nov. - 4 dec., Sydney, Australia, Vol. 6, pp. 2619-2622. [PDF] [PS]
C. Cucchiarini, H. Strik, F. de Wet & L. Boves (1998)
Assessment of Dutch pronunciation by means of automatic speech recognition technology: Automatic pronunciation grading for Dutch.
Proc. of the fifth Int. Conference on Spoken Language Processing (ICSLP'98), 30 nov. - 4 dec., Sydney, Australia, Vol. 5, pp. 1739-1742. [PDF] [PS]
C. Cucchiarini, H. Strik & L. Boves (1998)
Automatic pronunciation grading for Dutch.
Proc. of the ESCA workshop 'Speech Technology in Language Learning', Marholmen, Sweden, 25 - 27 May 1998, pp. 95-98. [PDF] [PS] [HTML]
C. Cucchiarini, H. Strik & L. Boves (1997)
Automatic evaluation of Dutch pronunciation by using speech recognition technology.
In: S. Furui, B.-H. Juang & W. Chou (eds.) Proc. of the 1997 IEEE workshop ASRU, Santa Barbara, pp. 622-629. [PDF] [PS] [HTML]
C. Cucchiarini, H. Strik & L. Boves (1997)
Using speech recognition technology to assess foreign speakers' pronunciation of Dutch.
Proc. of the Third int. symposium on the acquisition of second language speech: NEW SOUNDS 97, Klagenfurt, Austria, 8 - 11 Sept. 1997, pp. 61-67. [PDF] [PS] [HTML]
C. Cucchiarini, H. Strik & L. Boves (1997)
Automatic assessment of foreign speakers' pronunciation of Dutch.
Proc. of EuroSpeech'97, Rhodes, 22-25 Sept. 1997, pp. 713-716.

Presentations

Automatic assessment of second language learners' fluency.
Oral presentation T3TEL3 given at the 14th Int. Congress of
Phonetic Sciences (ICPhS-99), San Francisco, USA, Tuesday 3 August 1999. [PDF] [RTF] [TXT]
Automatic pronunciation grading for Dutch.
Oral presentation given at Ordinate Corporation, Menlo Park
(Silicon Valley), California, USA, Monday 9 August 1999. [PDF] [RTF] [TXT]
Computergestuurde Spreekvaardigheidstoets: automatische verwerking en scoring van antwoorden.
Presentation for A2RT at the Dept. of Language & Speech, University of Nijmegen, on Thursday 25 Nov. 1999. [PDF] [RTF] [TXT]
L2 pronunciation quality in read and spontaneous speech: expert ratings and objective measures
The 6th Int. Conference on Spoken Language Processing (ICSLP-2000), Beijing, China, 19-10-2000. [PPT] [PDF] [RTF] [TXT]
 

Links

* Automatic large-scale oral language proficiency assessment
    http://www.dsp.sun.ac.za/~trn/reports/dewet+vanderwalt+niesler_interspeech07.pdf
* CONCERNS WITH COMPUTERIZED ADAPTIVE ORAL PROFICIENCY ASSESSMENT
    http://llt.msu.edu/vol5num2/norris/default.html
* Computerized Oral Proficiency Assessment: Test Development and Test Selection
    http://www.allacademic.com/meta/p_mla_apa_research_citation/1/7/4/9/3/p174934_index.html

 
Last updated on 22-12-2008