The fundamental aim of this project is to boost productivity in natural language processing (NLP) research in the UK. At present a large amount of time is wasted by highly-skilled researchers who must continually reinvent the wheel when starting a new project or a new set of experiments. As in other fields, language processing work involves repetitive tasks that provide necessary infrastructure but do not directly advance our R&D. Unfortunately, whereas in other fields advanced computational tools are available that fulfil infrastructural needs (e.g. MatLab or Mathematica for mathematicians), no such general infrastructure is available in NLP.

Every PhD thesis or research grant from EPSRC (or the EC) typically begins its software development phase by defining a new set of tools and data structures for tasks such as:

• the management of the textual data to be processed;
• the storage and exchange of information about texts;
• the visualisation of the data (essential for efficient development and debugging);
• the comparison of the results of different configurations of software or with human-produced results;
• accessing resources such as lexicons and dictionaries that each has its own idiosyncratic format.

All of these tasks require robust, engineered software solutions, and all of them are key factors in the success of UK NLP research. The types of data structure typically produced and manipulated by NLP programs are large and complex, and without good tools to manage storage of, and allow succinct graphical viewing of, these structures, we work below our potential. At this stage in the progress of our field, no one should really have to write a tree viewing program for the output of a syntax analyser, for example, or even have to do significant work to get an existing viewing tool to process their data. While it is true that a number of specialised infrastructures exist, these are tied into particular theoretical commitments, for example attribute-value matrices evaluated under unification. Particularly at a time when statistical and hybrid statistical/symbolic work is so prevalent, these specialised systems cannot fulfil the role of a general NLP architecture.

In addition, many common language processing tasks have been solved to an acceptable degree by previous work and should be reused. Instead of writing a new part of speech tagger, or sentence splitter, or list of common nominal compounds, we should have available a store of reusable tools and data that can be plugged into our new systems with minimal effort. Such reuse is much less common than it should be, often because of installation and integration problems that have to be solved afresh in each case, and this is a major waste of research funding.

Building on previous work funded by the EPSRC, this project will eliminate the need for the majority of language processing workers to bother with infrastructure. We will provide the equivalent of Mathematica for language research and development. We will also make available a wide range of building blocks for common tasks (e.g. tagging) and to access common data resources (e.g. WordNet). The infrastructure and the building blocks will be WWW-enabled, and may reside on more than one machine as needed. A server in Sheffield will provide a high quality gateway to architectural services, language engineering building blocks and data resources.

Our aim, then, is to advance UK research in language processing, and enable us to preserve our leading role in the EC’s programmes, particularly at a time when developments on the Internet are driving investment in this area and have led to a number of new commercial developments. Sadly, a high proportion of the new companies and products appearing now are based in the US. Our work will contribute to easing the commercialisation of language technologies by providing an efficient, robust and well-documented data management substrate that can be delivered as part of applications (without any of the graphical development tools that a researcher needs). One advantage that we have over the US is our proximity to multi-lingual continental Europe: our infrastructure is both language-neutral and ideally suited to minimising integration problems in multi-site projects.

Last but not least, academics teaching computational linguistics, natural language processing and natural language engineering, and some parts of cognitive science, linguistics and modern languages can all benefit from easy-to-use graphical tools for manipulating language and language-related data structures.

Below we give comments on reviews of a previous version of this proposal, then section 1 describes our previous work in this area, and introduces the key personnel involved. Section 2 gives extensive technical detail of the work we will carry out in the project, beginning with a review of related previous work in the field (2.1) which motivates the broad stroke of our design (2.2), which is broken into three work packages. The technologies we plan to use are described (2.3). Section 3 gives planning details of the work programme, and section 4 additional relevant information for management and results exploitation. Section 5 gives references, and finally there is a Gantt chart of project packages and milestones.

A previous version of this proposal had largely positive reviews. We respond here to negative points that were raised.

The previous proposal included substantial additional research work in Information Extraction (IE) as part of our ongoing programme in that field. This was consistent with our previous EPSRC project, but was thought inadvisable by the review panel since it made the proposal too large and too obviously separable into architecture and IE parts. We have removed that element of the work in this version, and concentrated on the architecture alone.

One reviewer questioned the viability of completing all the work specified within the time limit, using the labour requested. There are two answers to this. First, we have designed the work programme with multiple deliverables and many milestone points that represent real advances. Should we fail to complete every single part of the programme we would still have made a major contribution to the field. Secondly, we have recently completed a similar-sized project (EPSRC GR/K25267), where we have completed or exceeded the original aims of the proposal, on time and within budget. Additionally, that project began with untrained junior staff, several of whom now, after 3 years involvement with the technology underlying this proposal, form one of the most highly skilled Language Engineering development teams. We feel that there is no question that we can complete the work we propose.

It was felt that we should provide more evidence of the uses to which GATE is being put – we have added to section 2.1.4 accordingly.

One reviewer wrote that 1) the work has no scientific basis; 2) asked why not use Java?; 3) asked how will we integrate with Office environments?; 4) said it may become obsolete quickly because commercial software moves so fast; 5) said it will never run fast enough to be useful in industry; 6) said too little time has elapsed to evaluate the success of the first version; 7) said that the proposal has far too much labour allocated. Taking these in turn:

1. Not science. The Council has a responsibility to Engineering and Science, and what we do, like all speech and language R&D, is often more the former than the latter. The role of this architecture has been, and will be, to enable substantial amounts of evaluable research (and in that sense scientific) at Sheffield and elsewhere, work it will be far harder to do without such underpinning. We offer a leap forward in such UK-based work. As further evidence of this, we are at present collecting code reuse figures for projects using GATE 1, which we will publish in scientific journals (as we have all our other results, whose referees thought they had merit).
2. Java? We already support Java (from version 1.5), and have based our design on many of the facilities available on that platform.
3. Office integration. We have planned for this, but space limitations and the belief that technical details were less interesting than the design squeezed it out of the original proposal. Please see the new section Technological Basis (2.3).
4. Obsolescence. Cunningham, the designated lead RA, has spent the last 6 years developing in C++, Tcl/Tk and Java in academia, and several years before that developing in COBOL in industry, and his experience suggests that in many cases academic software moves more quickly than commercial software. In any case, we are developing tools whose utility does not depend on the vagaries of current software fashion, but rather address research into fundamental questions of modelling human faculties using computation.
5. Speed. From the point of view of the developer, graphical RAD tools are always more resource hungry than applications for the end user (using MS Visual C++ on a 133Mhz Pentium with 32Mb RAM gives an example of slow-running commercial software!). From the point of view of delivering software based on the underlying data management substrate of the architecture, we fail to see how the design we have put forward (that uses conventional database software and the optimised data structures of the TIPSTER architecture committee, which included substantial commercial interests) is likely to be slow, and our reviewer gave no evidence to back up the assertion.
6. Too little evaluation time. It is now 18 months since the release of GATE 1.0, and we have included details of its take up and use. As noted under point 4) software moves quickly: 18 months ago Java was a little-known novelty language!
7. Too much labour. Another reviewer thought the programme too ambitious given the labour requested – see above. GATE 1 and the IE components delivered with it ran to 100,000 lines of code in 5 programming languages. Some industrial software process planning literature assumes 10 lines of fully debugged and documented code per programmer per day, which would have meant 27 person years to produce our system (which was in fact produced, under EPSRC funding, along with numerous academic papers and competing in two MUC evaluations etc., in 9 person-years).

VIE is the freely distributed counterpart of Sheffield's internal information extraction research system LaSIE (Large-Scale IE). Work on LaSIE has led to:

The Natural Language Processing group in the Computer Science Department (and ILASH: the Institute for Language, Speech and Hearing) is a major UK group in the field led by Yorick Wilks, who set it up on his return to the UK in 1993, and who will manage this project.  Major NLP R&D efforts in the recent past include:

III) Automatic lexicon construction: the CRL Lexical Knowledge Base (LKB) was derived as part of a US National Science Foundation grant (of which Wilks was principal investigator) from the LDOCE dictionary and the Collins Spanish-English bilingual dictionary. This resource was perhaps the first automatically derived lexical resource to have extensive practical application, and has been used by various groups (NYU, Brandeis, USC/ISI, CMU/CMT, the US Department of Defense, etc.) to extract information to analyse texts, within the PANGLOSS and TIPSTER ARPA projects.  The group is now a major contributor to the structural lexical resource projects EuroWordnet and PAROLE/SIMPLE funded by the EC. Wilks has a long history of collaboration with major publishers involved in the production of linguistic resources; particularly Longmans (where he has been on their linguistic committee for many years) CUP, Collins and OUP.  In New Mexico he set up the NSF-funded Consortium for Lexical Research.

The NLP/ILASH group hosted the AISB conference in 1994, and an EPSRC-DTI supported workshop on architectures for NLP in 1996, and an EPSRC-supported workshop on NLP evaluation in 1997.  It has world-wide affiliations, including official collaborations in the area with NEC and Hitachi Laboratories, and is the WWW host for international computational linguistics (ICCL-COLING) and the new UK Computational Linguistics initiative (CLUK).

Yorick Wilks has been, since 1993, Professor of Computer Science at the University of Sheffield and Director of ILASH, the Institute of Language, Speech and Hearing.  He received his doctorate from Cambridge University in 1968 for work on computer programs that process written English texts in terms of a theory later called "preference semantics": the claim that language is to be understood by means of a search for semantic patterns or "gists" in texts, combined with a coherence function over such structures that minimises effort in the analyser, and can make hypotheses about possible new senses of text words in context. He was a researcher at Stanford AI Laboratory, and then Professor of Computer Science and Linguistics at the University of Essex in England, before founding the Computing Research Laboratory in New Mexico, where he proposed and managed a range of large research projects in NLP, including information extraction (TIPSTER), lexicon development and belief and knowledge structures. He currently leads four EC and two EPSRC projects.  He has published numerous articles and six books in that area of artificial intelligence, of which the most recent are Artificial Believers (with Afzal Ballim) from Lawrence Erlbaum Associates (1991) and Electric Words: dictionaries, computers and meanings (with Brian Slator and Louise Guthrie) from MIT Press, (1996).  He is  on the EPSRC College of Computing, a Fellow of the American Association for Artificial Intelligence, on the boards of some fifteen AI-related journals, and on the International Committee on Computation Linguistics, the management group of CLUK, the SALT2 Committee and Longman's Linglex Committee.

Robert Gaizauskas is a lecturer in the Computer Science Department, University of Sheffield and has a DPhil from the University of Sussex in the area of computational logic.  He worked on the design and implementation of several information extraction systems, as well as the EPSRC-supported GATE architecture for text engineering with Wilks.  He is currently investigator on a BBSRC/EPSRC bioinformatics project to construct automatically a structured database of protein active site data from electronic journals in the field of molecular biology, is leading the project with Glaxo-Wellcome and Elsevier Science described above, and is involved in several European-funded projects to do with language engineering and evaluation of language technology.  He has published numerous papers, has acted as a project reviewer and rapporteur for the EC Language Engineering programme, has been involved in the EC Expert Advisory Group on Language Engineering Standards (EAGLES), is on the UK SALT steering committee, and has worked in industry for Fulcrum, a leading information retrieval company.

Progress in natural language engineering (LE) can benefit significantly from a generally acceptable and extensible "architecture": a software infrastructure with which researchers and developers can share, reuse and compare LE modules and resources. The success of the   US DARPA LE programmme has been evidence of this, and our GATE system has both benefited from, and contributed to that movement.  It has helped keep the UK at the forefront of LE world-wide.  Combinations of modules within such an architecture can perform a great range of research and industrial LE tasks.

An illustrative set of the sort of major "use cases" we expect for GATE2 would be: a dictionary publisher layers our common lexical resources model on top of their dictionary and makes it available on the WWW for limited queries with a view to encouraging researchers and generating extra sales of the unlimited version; the TIPSTER organisation (or an EU equivalent) makes its common module pool available for running remotely on their server, processing texts in databases on client machines; a supplier of information retrieval software (such as a large UK finance house) uses the interface to develop a name recognition component and uses the document manager with interface stripped to deliver the component; academics teaching language processing or language learning use the graphical facilities and rapid prototyping capabilities to structure their classes; researchers in etymology use it to collect statistics on word evolution from online newspapers, without copying all the web pages; an EC R&D project uses it as the backbone for developing term substitution and name transliteration add-ins for the Han character word-processing market.

An architecture in this proposal is a macro-level organisational pattern for the components and data resources that make up a language processing system; development environments add graphical tools to access the services provided by the architecture.  The GATE architecture does this using three subsystems: (1) GDM, the GATE Document Manager; (2) GGI, the GATE Graphical Interface; (3) CREOLE, a Collection of REusable Objects for Language Engineering. GDM manages the information about texts produced and consumed by NLP processes; GGI provides visual access to this data and manages control flow; CREOLE is the set of resources so far integrated with GATE.

Existing systems that provide software infrastructure for NLP can be classified as belonging to one of three types according to the way they manage information about texts:

• additive, or markup-based: information produced is added to the text in the form of markup, e.g. SGML;
• referential, or annotation-based: information is stored separately with references back to the original text, e.g. the TIPSTER architecture;
• abstraction-based: the original text is preserved in processing only as part of an integrated data structure that represents information about the text in a uniform, theoretically-motivated, model, e.g. attribute-value structures in the ALEP system.

2.1.2 The GATE graphical interface (GGI)

One of the key benefits of adopting an explicit architecture for data management is that it becomes straightforward to add graphical interface access to architectural services and data visualisation tools, and such a layer is our second pillar: GGI, the GATE graphical interface. GGI has functions for creating, viewing and editing collections of documents which are managed by the GDM and that form the corpora which LE modules and systems in GATE use as input data. It also has facilities to display the results of module or system execution, i.e. new or changed annotations associated with the document. These annotations can be viewed either in raw form, using a generic annotation viewer, or in an annotation-specific way, if special annotation viewers are available. For example, named entity annotations which identify and classify proper names (e.g. organisation names, person names, location names) are shown by colour-coded highlighting of relevant words; phrase structure annotations are shown by graphical presentation of parse trees; coreference chains are shown through highlighting linked phrases.

2.1.3 A developing Collection of REusable Objects for Language Engineering (CREOLE)

The process of integrating modules into GATE has been automated to a large degree and can be driven from the interface. The developer is required to produce some C++, Tcl or Java code that uses the GDM TIPSTER API to get information from the database and write back results. The underlying module can be in C/C++, Java or Tcl, or be an external executable written in any language (the current CREOLE set includes Prolog, Lisp and Perl programs, for example).

2.1.4 Users of GATE 1

We maintain a database of the groups and individuals who are using GATE for a wide variety of tasks (and the feedback we have received from these users has been incorporated in the content of this proposal), for example:

GATE 2 will take advantage of the prevalence of object-centred graphical interfaces to provide a look-and-feel that follows de-facto standards and hence has a low learning overhead. For example, there will be object-specific management tools for all domain objects (systems, modules, collections, documents, annotations). These tools will provide tabbed views, drag and drop, flexible customisation (e.g., user-defined filters on annotations will be displayed in separate tabs). In general, the aim is to make GATE look like a desktop incorporating all these management tools, so that the user can easily manipulate systems and documents and also inspect the results associated with each document. An easy-to-learn, easy-to-use interface will increase the willingness of researchers to adopt GATE as a tool and will promote its use as a teaching tool (several users are already using GATE for teaching and there has been considerable interest expressed by colleagues at Sussex, Edinburgh and Lancaster in extending its utility for teaching). The interface will support Unicode, and will allow editable documents with automatic annotation offset recalculation (via a gap buffer). We will investigate the problem of chaining updates to annotation structures during editing. GGI will be a set of Java Beans, so developers will be able to build their own customised interfaces (see 2.1.3).

This new subsystem will:

• encapsulate a range of alternative control regimes, e.g. for distributed operation of cooperating autonomous agents;
• consolidate the algorithmic resource integration mechanisms that currently live in GGI and GDM and extend those mechanisms to support distributed tools;
• add integration and management mechanisms for linguistic resources such as lexicons.

At present, services such as the Lexical Data Consortium, the European Language Resources Association, and the Stuttgart Registry are static servers that require the user to download source code or data and install it on their own machines. We will develop an Active CREOLE Server with which researchers can assemble systems from components that may be running on a mixture of their own site, the server site (Sheffield) and at others' sites. The overheads involved in integrating diverse components will then be reduced to an absolute minimum. Where sites wish to retain licensing control access can be password-based, or forwarded to a server local to the developer.

• each resource has its own representation syntax and corresponding access mode: relational databases in the case of CELEX, a custom C API or Prolog representation in Wordnet, SGML in the case of the British National Corpus;
• resources must be installed locally to be usable (and precisely how this happens, what operating systems are supported etc., varies from site to site).

The consequences of the first bullet are that although linguistic resources normally share some structure in common (e.g. at the most obvious level, lexicons are organised around words and word strings) this commonality is wasted when using a new resource, since the developer has to learn everything afresh each time, and  work which seeks to investigate or exploit commonalities between resources (e.g. to link several lexicons to an ontology) has first to build a layer of access routines on top of each resource. So, for example, if we wish to do task-based evaluation of lexicons by measuring the relative performance of an information extraction system with different instantiations of the lexical resource, we might end up writing code to translate several different resources into SQL or SGML. The consequence of the second bullet above is that there is no way to "try before you buy": no way to examine a data resource for its suitability for your needs before licensing it. Correspondingly, there is no way for a resource provider to give limited access to their products for advertising purposes, nor gain revenue through piecemeal supply of sections of a resource.

Most of this proposal can be viewed as an extended requirements analysis and high-level design for a long-term infrastructure for language work. Low-level design would be inappropriate at this stage. However, it would also be foolhardy to embark on this type of software development without an idea of the technological risks involved (see e.g. the section on process in [Fow97] for good arguments for doing technology identification an early stage). This section sketches the key technologies we provisionally plan to use for our implementation.

This technology is new, but we have run preliminary experiments with each part of the jigsaw, and are confident of the feasibility of this implementation route.

Integration of distributed document manager from CRL.

Retargeting of existing C++, Tcl and Java APIs on new document manager (to ensure backwards compatibility with GATE 1)

Extension of document manager to speech and multi-media

Addition of information retrieval classes to document manager

Extension of SGML input filter to other document types (including HTML, XML, email and RTF).

Development of new object-centred Unicode interface with editable documents

Conversion of GATE 1 annotation comparison, annotation creation and annotation visualisation tools to new interface

Provision of interface support for dialogue processing

Creation of an applet version of the interface for WWW-based access to services and resources

Resource manager subsystem with optional autonomous agent control executive

Extension of existing integration automation to distributed CREOLE objects

Integration of an existing UK information retrieval system (possibly Okapi)

Integration of other UK NLP components

Integration of the Sheffield/Cambridge Abbot speech recognition system

Integration of the CRL MT system

Integration of ViewGen

Integration of public domain NLP components from e.g. the Stuttgart registry

WP 3: Active CREOLE server

Deployment of active CREOLE web server (both API-based access and browser access)

Development of linguistic resource object model

Integration of linguistic resources including EuroWordnet, WordNet, CELEX, the ANLT lexicon, CRL LDB and Microkosmos.

Population of CREOLE server with modules and resource

Web demonstrator for IE systems

Each work package proposed makes advances in Language Engineering (LE) technology and facilitates further advances by the UK R&D Community. The GATE2 NLP architecture is itself novel, and we hope the best available world-wide; the integrated modules we provide will be a rich and novel platform for experimental and technological advance throughout the community, including our own (Sheffield) programme in LE and computational linguistics.

The value of GATE1 to a wide range of R&D groups has already been demonstrated, and GATE2 will advance that standard, helping keep UK work in the front line without the need for direct participation in US competitions, useful as that is. GATE2 will assist system developers both in research and in any branch of the language industries (especially electronic publishing and news dissemination), speeding the integration of functionalities and their testing. This will be evaluated and tested with our collaborators (see attached letters of support).

Apart from the standard academic dissemination channels of conferences and journal/book publication (including web and archive publication), we shall continue with specialised workshops, both those we shall organise ourselves, and those we have in the past been invited to join, such as DTI/EPSRC sponsored industrial forums and the Unicom seminar series for industry. GATE is already widely disseminated, and it seems likely at the time of writing that US DARPA may make it an interim NLP architecture standard, which would entrench its R&D position world wide; it has also been used as the basis of a range of EU language engineering projects. We shall continue to publicise it, though without offering maintenance, and will set up a user group and some form of industrially orientated workshop/forum for GATE users, and those interested, during the first year of the project. We have considerable experience in organising workshops of this sort. For both GATE and the IE system, we shall maintain our direct links with US and EU standards-setting bodies, continuing to argue that standards are vital but must be based on systems that are widely acceptable to users. We also participate in ELSE, the EU NLP evolution standards project, and hope to make GATE2 the platform for Europe-wide evaluation in NLP, thus linking EU and US activities to the overall benefit of the UK position as a fulcrum in the field.

We have a developed set of web pages advertising aspects of GATE and our IE work, through our department and through ILASH and we also maintain the linked CLUK and ICCL web sites. We also produce industrially-orientated publicity materials, and receive industrial solicitations as a result of these (and actively solicit others from relevant firms). With them we seek to spread a culture of adapting/customising LE systems to particular industrial requirements. A key feature of the current proposal is that we shall set up a publicly available WWW demo of GATE 2, with as many modalities as proves feasible.

The project will be managed by a team led by Yorick Wilks as principal investigator: Rob Gaizauskas will manage requirements capture and Hamish Cunningham system development. Wilks has managed large NLP projects for UK and US agencies, and NLP architecture projects since the beginning of the US TIPSTER project; he was the PI of GATE1. Gaizauskas is widely experienced from the Sussex POETIC project and has designed most of the EPSRC-supported LaSIE system. Cunningham has gained extensive experience with the implementation and deployment of GATE1 and its use in major EU projects. The Sheffield ILASH/CS NLP group has a tight and experienced management structure with its own research coordinator, systems administrator and secretary who ensure delivery of modules, conformance to reporting and delivery deadlines, the archiving of software and the organisation of workshops and related events. The NLP group maintains a seminar series and a set of linked study groups to keep abreast of the literature and to train its research students. The group maintains its own monthly newsletter that also sets out deadlines and milestones: each research project also has its own weekly meeting reporting to the NLP group weekly meeting. We hold regular and frequent meetings with industrial collaborators, whom we also invite to join our departmental Industrial Associates Group.

The proposal requests two RAs: Hamish Cunningham, with substantial engineering experience to lead GATE2 development and module integration, and one junior RA to program on GATE2. Each of these would be fully committed. Each would require a state of the art workstation, since we have no other source for these (our industrial collaborators not being hardware manufacturers), and we also need hardware to support the Active CREOLE Server.

The travel budget represents normal participation at a subset of major conferences to disseminate results (IJCAI, ECAI, AISB, ACL, COLING) with some provision for participation in DARPA events (like MUC) since they are essential if we are to remain a focal part of the international community by participating in the DARPA steering committee.

Adv96  Advanced Research Projects Agency. Proceedings of the TIPSTER Text Program (Phase II). Morgan Kaufmann, California, 1996.

AHG97  S. Azzam, K. Humphreys, R. Gaizauskas, H. Cunningham, and Y. Wilks. A Design for Multilingual Information Extraction (poster). In IJCAI-97, 1997.

BMSW97  D. Bikel, S. Miller, R. Schwartz, and R. Weischedel. Nymble: a High-Performance Learning Name-finder. In Proceedings of the Fifth conference on Applied Natural Language Processing, 1997.

Boo94 G. Booch. Object-Oriented Analysis and Design 2nd Edtn. Benjamin/Cummings, 1994.

Bri95  E. Brill. Transformation-Based Error-Driven Learning and Natural Language. Computational Linguistics, 21(4), December 1995.

BW91  A. Ballim and Y. Wilks. Artificial Believers. Lawrence Erlbaum Associates, Hillsdale, New Jersey, 1991.

Chr94  O. Christ. A Modular and Flexible Architecture for an Integrated Corpus Query System. In Proceedings of the 3rd Conference on Computational Lexicography and Text Research, 1994.

CFB94 H. Cunningham, M. Freeman, and W.J. Black. Software Reuse, Object-Oriented Frameworks and Natural Language Processing. In Proceedings of the conference on New Methods in Natural Language Processing (NeMLaP-1), Manchester, 1994.

CGW95  H. Cunningham, R. Gaizauskas, and Y. Wilks. A General Architecture for Text Engineering (GATE) – a new approach to Language Engineering R&D. Technical Report CS – 95 – 21, Department of Computer Science, University of Sheffield, 1995. Also available as http://xxx.lanl.gov/ps/cmp-lg/9601009.

CHGW96  H. Cunningham, K. Humphreys, R. Gaizauskas, and Y. Wilks. TIPSTER-Compatible Projects at Sheffield. In Advances in Text Processing, TIPSTER Program Phase II. DARPA, Morgan Kaufmann, California, 1996.

CHGW97a  H. Cunningham, K. Humphreys, R. Gaizauskas, and Y. Wilks. GATE – a TIPSTER-based General Architecture for Text Engineering. In Proceedings of the TIPSTER Text Program (Phase III) 6 Month Workshop. DARPA, Morgan Kaufmann, California, May 1997.

CHGW97b  H. Cunningham, K. Humphreys, R. Gaizauskas, and Y. Wilks. Software Infrastructure for Natural Language Processing. In Proceedings of the Fifth Conference on Applied Natural Language Processing (ANLP-97), March 1997. Available as http://xxx.lanl.gov/ps/9702005.

Cun94 H. Cunningham, M. Freeman, and W.J. Black. Software Reuse, Object-Oriented Frameworks and Natural Language Processing. In Proceedings of the conference on New Methods in Natural Language Processing (NeMLaP-1), Manchester, 1994.

CWG96a  H. Cunningham, Y. Wilks, and R. Gaizauskas. GATE – a General Architecture for Text Engineering. In Proceedings of the 16th Conference on Computational Linguistics (COLING-96), Copenhagen, August 1996.

CWG96b  H. Cunningham, Y. Wilks, and R. Gaizauskas. Software Infrastructure for Language Engineering. In Proceedings of the AISB Workshop on Language Engineering for Document Analysis and Recognition, Brighton, U.K., April 1996.

CWG96c  H. Cunningham, Y. Wilks, and R. Gaizauskas. New Methods, Current Trends and Software Infrastructure for NLP. In Proceedings of the Conference on New Methods in Natural Language Processing (NeMLaP-2), Bilkent University, Turkey, September 1996. Also available as http://xxx.lanl.gov/ps/cmp-lg/9607025.

DAH97  D. Day, J. Aberdeen, L. Hirschman, R. Kozierok, P. Robinson, and M. Vilain. Mixed-Initiative Development of Language Processing Systems. In Proceedings of the 5th Conference on Applied NLP Systems (ANLP-97), 1997.

ELNP97  D. Estival, A. Lavelli, K. Netter, and F. Pianesi, editors. Computational Environments for Grammar Development and Linguistic Engineering. Association for Computational Linguistics, July 1997. Madrid, ACL-EACL'97.

Fow97 M. Fowler. UML Distilled: Applying the Standard Modelling Language. Addison-Wesley, 1997.

Eva97 R. Evans. SEAL: Structural Enhancement of Automatically-acquired Lexicons. http://www.itri.brighton.ac.uk/projects/seal. 1997

GHAW97 R. Gaizauskas, K. Humphreys, S. Azzam, and Y. Wilks. Concepticons vs. Lexicons: An Architecture for Multilingual Information Extraction. In M.T. Pazienza, editor, Proceedings of the Summer School on Information Extraction (SCIE-97), pages 28—43. Springer-Verlag, 1997.

GCW96  R. Gaizauskas, H. Cunningham, Y. Wilks, P. Rodgers, and K. Humphreys. GATE – an Environment to Support Research and Development in Natural Language Engineering. In Proceedings of the 8th IEEE International Conference on Tools with Artificial Intelligence (ICTAI-96), Toulouse, France, October 1996.

GH96  R. Gaizauskas and K. Humphreys. Using verb semantic role information to extend partial parses via a co-reference mechanism. In J. Carroll, editor, Proceedings of the Workshop on Robust Parsing, pages 103–113, Prague, Czech Republic, August 1996. European Summer School in Language, Logic and Information.

GH97a  R. Gaizauskas and K. Humphreys. Quantitative Evaluation of Coreference Algorithms in an Information Extraction System. In S. Botley and T. McEnery, editors, Discourse Anaphora and Anaphor Resolution. In Press. Also available as CS – 97 – 19.

GH97b R. Gaizauskas and K. Humphreys. Easing a semantic network for information extraction. Journal of Natural Language Engineering. In Press

Got98 Y. Gotoh, S, Renals, R. Gaizauskas, G. Williams, and H. Cunningham. Named Entity Tagged Language Models for LVCSR. Technical Report CS-98-05, Department of Computer Science, University of Sheffield, 1998.

GR97a  R. Gaizauskas and A.M. Robertson. Coupling information retrieval and information extraction: A new text technology for gathering information from the web. In Proceedings of RIAO'97, Montreal, 1997.

GR97b  R. Gaizauskas and P. Rodgers. NL Module Evaluation in GATE. In Proceedings of the SALT club workshop on Evaluation in Speech and Language Technology, 1997.

Gri96  R. Grishman. TIPSTER Architecture Design Document Version 2.2. Technical report, DARPA, 1996. Available at http://www.tipster.org/.

Gri97  R. Grishman. Information Extraction: Techniques and Challenges. In Information Extraction: a Multidisciplinary Approach to an Emerging Information Technology, Springer 1997.

GS96  R. Grishman and B. Sundheim. Message understanding conference - 6: A brief history. In Proceedings of the 16th International Conference on Computational Linguistics, Copenhagen, June 1996.

GWH95  R. Gaizauskas, T. Wakao, K Humphreys, H. Cunningham, and Y. Wilks. Description of the LaSIE system as used for MUC-6. In Proceedings of the Sixth Message Understanding Conference (MUC-6). Morgan Kaufmann, California, 1995.

Har91  P. Harrison. Evaluating Syntax Performance of Parsers/Grammars of English. In Proceedings of the Workshop on Evaluating Natural Language Processing Systems, ACL, 1991.

Hum97a W.S. Humphreys. Introduction to the Personal Software Process. Addison-Wesley, 1997.

Kam97  M. Kameyama. Information Extraction across Linguistic Boundaries. In AAAI Spring Symposium on Cross-Language Text and Speech Processing, Stanford University, 1997.

KE95 Kilgarriff, A. and Evans, R., MRDs, Standards and How to Do Lexical Research Proceedings of the Language Engineering Conference, London, October 1995.

KGW94 A. Krotov, R. Gaizauskas and Y. Wilks. Acquiring a stochastic context-free grammar from the penn treebank. In Proceedings of the Third International Conference of the Cognitive Science of Natural Language Processing (CSNLP94), pages 79–86, Dublin, July 1994.

Lee95 M. Lee. A goal ascription based approach to conversational implicature. In Proceedings of the 14th Workshop of the UK Planning and Scheduling Special Interest Group, Essex, 1995.

LCB+97 D. Levy, R. Catizone, B. Battacharia, A. Krotov and Y. Wilks. CONVERSE: A Conversational Companion. In Proceedings of the 1^st International Workshop on Human-Computer Conversation. 1997

LW96  M. Lee, Y. Wilks. An ascription-based approach to speech acts. Proceedings of the 16th Conference on Computational Linguistics (COLING-96), Copenhagen, 1996.

LW96a M. Lee and Y. Wilks. An ascription-based approach to speech acts. In Proceedings of the 16th Conference on Computational Linguistics (COLING-96), Copenhagen, 1996.

LW96b M. Lee and Y. Wilks. Speech acts on demand. In Proceedings of the 5th International Pragmatics Conference, Mexico City, 1996.

LW97  M. Lee, Y. Wilks. Eliminating deceptions and mistaken belief to infer conversational implicature. In IJCAI-97 workshop on Conflict, Co-operation and Collaboration in Dialogue Systems, Tokyo, 1997.

MBT97  D. McKelvie, C. Brew, and H. Thompson. Using SGML as a Basis for Data-Intensive NLP. In Proceedings of the fifth Conference on Applied Natural Language Processing (ANLP-97), Washington, DC, 1997.

MF97  A. Mikheev, S. Finch. A Workshop for Finding Structure in Text. In Fifth Conference on Applied NLP (ANLP-97), Washington, DC, 1997.

Mor96  R.G. Morgan. An architecture for user defined information extraction. Technical Report 8/96, dept. Computer Science, University of Durham, 1996.

RGHC97  P.J. Rodgers, R. Gaizauskas, K. Humphreys, and H. Cunningham. Visual Execution and Data Visualisation in Natural Language Processing. In IEEE Visual Language, Capri, Italy, 1997.

RHR96  T. Robinson, M. Hochberg, and S. Renals. The use of recurrent networks in continuous speech recognition. In C. H. Lee, K. K. Paliwal, and F. K. Soong, editors, Automatic Speech and Speaker Recognition – Advanced Topics, chapter 10, pages 233–258. Kluwer Academic Publishers, Amsterdam, 1996.

RMB94  S. Renals, N. Morgan, H. Bourlard, M. Cohen, and H. Franco. Connectionist probability estimators in HMM speech recognition. IEEE Transactions on Speech and Audio Processing, 2:161–175, 1994.

Sch94  J. Schütz. Developing Lingware in ALEP. ALEP User Group News, CEC Luxembourg, 1(1), October 1994.

Sim94  N. K. Simkins. An Open Architecture for Language Engineering. In First CEC Language Engineering Convention, Paris, 1994.

Thu97  G. Thurmair. Information extraction for intelligence systems. In Natural Language Processing: Extracting Information for Business Needs, pages 135–149, London, March 1997. Unicom Seminars Ltd.

TM96  H. Thompson and D. McKelvie. A Software Architecture for Simple, Efficient SGML Applications. In Proceedings of SGML Europe '96, Munich, 1996.

TWY96  Y. Takemoto, T. Wakao, H. Yamada, R. Gaizauskas, and Y. Wilks. Description of the NEC/Sheffield System Used for MET Japanese. In Proceedings of the TIPSTER Phase II Workshop, 1996.

VD96  M. Vilain and D. Day. Finite-state phrase parsing by rule sequences. In Proceedings of COLING-96, 1996.

WB96 Y. Wilks and T. By. Procedural conventions for reference resolution in belief environments. Submitted to Cognitive Science, 1998.

WBB91 Y. Wilks, J. Barnden, and A. Ballim. Belief ascription, metaphor and intensional identity. Cognitive Science, 15(1), 1991.

WBW91 Y. Wilks, J. Barnden, and J. Wang. Your metaphor or mine: Belief ascription and metaphor interpretation. In Proceedings of IJCAI-91, Sydney, Australia, 1991.

WG97  Y. Wilks and R. Gaizauskas. LaSIE jumps the GATE. In Natural Language Processing: Extracting Information for Business Needs, UNICOM, 1997.

WGS95  Y. Wilks, L. Guthrie, and B. Slator. Electric Words: Lexicons, Dictionaries and Meanings. MIT Press, Cambridge, MA, 1995.

WGW96  T. Wakao, R. Gaizauskas, and Y. Wilks. Evaluation of an algorithm for the recognition and classification of proper names. In Proceedings of the 16th International Conference on Computational Linguistics (COLING96), pages 418–423, Copenhagen, 1996.

Wil96 Y. Wilks. Senses and texts. Computers and Humanities, 1997.

Zaj97b R. Zajac. An Open Distributed Architecture for Reuse and Integration of Heterogeneous NLP Components. In Proceedings of the 5th conference on Applied Natural Language Processing (ANLP-97), 1997.