Proposal Number : IST-1999-10651
Acronym : MUMIS
Title : Multimedia Indexing and Searching Environment

Key Action : KA III
Research Area : Human Language Technologies
Action line : III.4.1 (Multilinguality in Digital Content and Services)

MUMIS will develop base technology, demonstrated within a laboratory prototype, to support automated multimedia indexing and to facilitate search and retrieval from multimedia databases in specific domains. The project will demonstrate that innovative technology components can operate on multilingual, multisource, and multimodal information and create a meaningful and queryable database with video content from the soccer domain.

MUMIS will develop technology for the indexing and retrieval of data from different media sources (textual documents, radio and television programmes) for the selected domain (soccer)

MUMIS will develop technology for automatic indexing of multimedia programme material, including:

• Data capture software, to digitise audio and video recordings, standardise text files, extract text from subtitle files
• Speech recognition software, to extract text elements from audio files and video soundtracks
• Text annotation (information extraction) tools to extract annotations from texts
• Fusion software (merging tool) to combine semantically identical annotations taken from different media and in different languages (Dutch, English, German).

• Develop a user interface (in Dutch, English, German and Swedish) to enable professional users to query the database, by selecting from menus based on annotations, metadata, and so on, and to view video fragments which satisfy the query.

From a technological perspective, MUMIS is one of the first multimedia indexing projects which:

• uses multimedia and multilingual information sources;
• combines topic triggering by information retrieval (IR) with information extraction (IE) based on more detailed linguistic processing;
• carries out multimedia indexing by applying information extraction to a well-delineated domain and using already existing information as constraints;
• uses and extends advanced language technology to automatically create formal annotations for multimedia content;
• merges information from many sources to improve the quality of the annotation database;
• combines database queries with direct access to multimedia fragments over the web, to access distributed servers and using non-proprietary media streaming methods.

In terms of economic benefits, MUMIS will make access to video archives more effective and cheaper for professional users, reducing the cost of programme making. When applied beyond the professional market, MUMIS will make a contribution to improving employment, because it will help to launch a completely new line of electronic business: the sale of the contents of the enormously large and ever growing archives of digital multimedia programmes.

• Project Coordinator

• Project Participants