Human Language Technology for Automatic Annotation and Indexing of Digital Library

Content

Kalina Bontcheva and Hamish Cunningham

Department of Computer Science, University of Sheffield 211 Portobello St, Sheffield, UK S1 4DP

                      {kalina,hamish}@dcs.shef.ac.uk

Abstract. This demo will present a set of domain-independent and customisable Human Language Technology (HLT) tools and the way they were applied for annotating 18th century OldBailey proceedings and indexing multimedia content. This demo accompanies the paper with the same title.

1 Overview

Each of these digital library applications posed a unique challenge: the court trials required adapting the language processing components to the non-standard written conventions of 18th century English, while the football collection presented the challenge of indexing material in multiple modalities - video, audio, semi-structured documents, and free text (newspaper reports). In both cases, we used as a basis our robust and customisable named entity recogniser, which comes as part of ANNIE - A Nearly New Information Extraction system (freely available as part of the GATE system: http://gate.ac.uk). In OldBaileyIE we also used the graphical environment of GATE, which allows humans to verify and correct the automatically annotated documents.

First we will demonstrate the set of language technology tools and how they can be used to annotate named entities in documents (e.g., person names, organisations, locations, dates, money amounts, percents). These tools are freely available for research and commercial use and are also made customisable, so they can be adapted to new domains and applications.

Next we will demonstrate the automatic annotation of the 18th century Old-Bailey texts and present the visual environment used for annotation error correction. We will also demonstrate how this environment can be used for manual annotation, independent of the language technology tools.

Finally we will demonstrate how multimedia content can be indexed and searched using human language technology. The demonstrator is in the football domain, where video and textual sources about matches are indexed and can be accessed via a user-friendly interface.

These tools are capable of supporting multiple languages through Unicode. For further details papers from the GATE publications page:

http://gate.ac.uk/gate/doc/papers.html.