Modeling Pronunciation Variation for Automatic Speech Recognition

One of the difficulties continuous speech recognition is still facing is the variation which occurs due to pronunciation variation. Problems occur both during training and during recognition. During the training phase after an automatic segmentation has been done, parts of the signal can be assigned to incorrect phonemes. Because of this the trained phone models will be contaminated with information belonging to other phones. During the testing phase the problem arises that the match between the spoken signal and the canonical pronunciation form in the lexicon might not be optimal. This especially plays a role in spontaneous speech, because the realized pronunciations often vary immensely from each other, and therefore also from the canonical form in the lexicon.

The first aim of this project is to model pronunciation variation in Dutch in order to improve the performance of a spoken dialogue system. Second, the development of an aid for phonetic transcription and segmentation of large amounts of speech. In order to achieve these goals, two approaches are followed the top-down method and the bottom-up method. In this case "top" is to be interpreted as the words and "bottom" as the speech signals.

The top-down approach is more of a linguistic approach. Phonological knowledge about Dutch is applied by automatically generating pronunciation variants according to the known rules. Subsequently, the variants are matched to the signals. By explicitly modeling the pronunciation variation the recognizer's performance is expected to improve.

The bottom-up method is more technical and focusses mainly on the acoustic signal. By analyzing the acoustic information in the signal pronunciation variation is looked for and then modeled. Effort is also put into improving the manner in which acoustic modeling is done.

For more information on the results obtained in this research, see below (references).