\documentclass[10pt,a4paper]{article}
\usepackage{lrec2006}
\usepackage{qtree,pict2e}
\usepackage{url,moreverb,hhline,graphicx,lmodern}
\usepackage[british]{babel}
\usepackage[T1]{fontenc} % edh in BibTeX
\newcommand{\slug}{\hspace{1.0em}}
%%%%%
% Message-ID: <160220101402163735%gregorio@math.unipd.it>
\renewcommand{\thesection}{\arabic{section}}
\renewcommand{\thesubsection}{\thesection.\arabic{subsection}}
\renewcommand{\thesubsubsection}{\thesubsection.\arabic{subsubsection}}
\makeatletter
\renewcommand*{\@seccntformat}[1]{%
  \csname the#1\endcsname
  \ifcsname adddot@#1\endcsname.\fi\quad
}
\let\adddot@section=T
\let\adddot@subsection=T
\let\adddot@subsubsection=T
\makeatother
\urldef\mails\url|{a.funk,i.roberts,w.peters}@dcs.shef.ac.uk|
\selectlanguage{british}
%%%%%  max. 4 pages
\title{Implementing a Variety of Linguistic Annotations through a Common Web-Service Interface}
\name{Adam Funk, Ian Roberts, Wim Peters}
\address{Department of Computer Science\\
  University of Sheffield\\
  Regent Court, Sheffield, S1~4DP, UK\\
  \mails\\}
%
\abstract{We present a web service toolkit and common client for a series of
  natural language processing (NLP) services as a contribution to CLARIN'S
  European Demonstrator.  We have also deployed and tested several natural
  language processing and information extraction services for English and
  propose to develop further compatible services using resources for other
  languages.}
%%%%%
\begin{document}
\maketitleabstract
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Introduction}
\label{sec:intro}
%%%%%
An important goal of CLARIN is to make language resources and technology
available to humanities researchers and other end users, especially through web
services.  The European Demonstrator~\cite{demonstrator} prototype system will
integrate a number of resources and services available from the participating
institutions.  We present here a web service toolkit and client, along with the
implementation of a series of NLP and IE services, as a contribution to that
system.
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Web service implementation}
%%%%%
Our CLARIN services are standard SOAP web services.  They take their input as
binary data (as a MIME attachment to the SOAP message, according to the MTOM
specification)---though the services are intended to process text they can
handle input in many formats including XML, HTML and PDF, extracting the text
from the source data using the format handling mechanism provided by GATE.  The
service loads the input data into a GATE \emph{Document} object, then processes
that Document using a GATE \emph{DocumentProcessor} (typically a wrapper around
a saved GATE application), and returns its output as XML data (any valid XML
element is allowed for versatility).  (Please refer to the GATE manual
\cite{GateManual2010} and API
documentation\footnote{\url{http://gate.ac.uk/documentation.html}} for details
of the GATE library.)


All the services share a common WSDL interface as their inputs and outputs are
the same; only the underlying GATE application needs to vary between services.
The standard interface simply specifies the output as any XML in any namespace,
and the implementation does not restrict the XML that the underlying application
can produce.  However if the output types for a specific service are known and
there is a suitable W3C XML Schema available then there is the option to use a
custom WSDL for that service with the more constraining schema included, which
may be beneficial for certain types of client.


The various components making up the service implementation are configured using
the Spring framework, making it simple to slot in alternative
\emph{DocumentProcessor} implementations for different services without changes
to the code.  The aspect-oriented programming tools provided by Spring are used
to allow pooling of several identical DocumentProcessors, to support multiple
concurrent web service clients.  The web service layer is provided by the Apache
CXF toolkit, which itself uses Spring extensively and thus was a good fit with
the Spring-driven architecture adopted for the business logic.


This toolkit will work easily for any GATE application, typically a
\emph{SerialAnalyserController} or \emph{ConditionalSerialAnalyserController}
(corpus pipeline); furthermore, with suitable modification in the Spring beans
configuration, it can use any class that \emph{implements
  DocumentProcessor}---in effect, any class that can analyse a single GATE
Document (or a GATE \emph{Corpus} containing one Document) and produce any valid
XML document (the root element of which which we treat as the result and embed
in the SOAP response).  Each web service provides a WSDL file available from the
server and offers three methods:
%%
\begin{itemize}
\item \emph{process} send only the document content as a \url{byte[]};
\item \emph{processWithURL} also sends the document URL---the GATE
  \emph{Factory} will take the filename into consideration when instantiating
  the Document (to distinguish PDFs properly, for example);
\item \emph{processWithParams} sends a parameter list, which allows the client
  to specify the original URL, encoding, and mime type (this method allows the
  greatest flexibility).
\end{itemize}
%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Services currently available}
%%%%%
We have implemented the following services so far, making use of standards which
we have worked with in previous projects (particularly
LIRICS\footnote{\url{http://lirics.loria.fr/}},
SEKT\footnote{\url{http://www.sekt-project.com/}}, and
MUSING\footnote{\url{http://www.musing.eu/}}).
%%
\begin{itemize}
\item The \emph{annie-alpha} service runs the ANNIE \cite{Cun02b} named-entity
  recognition and orthographic co-reference pipeline and returns the fully
  annotated document in GATE XML format.  The file saved by the client contains
  ANNIE's output in the default AnnotationSet and the input document's HTML or
  XML mark-up in the ``Original markups'' AnnotationSet.
%%
\item The \emph{maf-en} service runs GATE's sentence-splitter, tokenizer,
  POS-tagger and morphological analyser (lemmatizer) for English, and returns an
  XML document containining the morphosyntactic information according to the MAF
  \cite{maf} standard.
%%
\item The \emph{chunking-synaf-en} service runs GATE's sentence-splitter,
  tokenizer, POS-tagger, and NP and VP chunkers \cite[\S17]{GateManual2010} for
  English, as well as a simple PP chunker, to produce annotations as shown in
  Figure~\ref{fig:chunking-spans} (where \emph{VG} means verb group).  The
  application then constructs a simple syntactic tree for each sentence based on
  simple containment (each phrase or token annotation is a constituent of the
  smallest sentence or phrase annotation containing it), as shown in
  Figure~\ref{fig:chunking-tree}, and returns an XML document according to the
  SYNAF \cite{synaf} standard.
  %%
  \begin{figure}[tbp]
    \begin{center}
      \includegraphics[width=0.95\columnwidth]{chunking-spans.png}
    \end{center}
    \caption{Spans of syntactic elements produced within the chunking service}
    \label{fig:chunking-spans}
  \end{figure}
  %%
  \begin{figure*}[tbp]
    \small
%    \begin{center}
%       \Tree [.Sentence [.NP [.Token This ] [.Token toolkit ] ]  %
%       [.VP [.Token will ] [.Token work ] ]  [.Token easily ]  %
%       [.PP [.Token for ] [.NP [.Token any ] %
%       [.Token GATE ]  [.Token application ] ] ] %
%       [.Token X ]       ]
      \Tree [.Sentence \qroof{This toolkit}.NP \qroof{will work}.VP easily %
      [.PP for \qroof{any GATE application}.NP ] \texttt{.} ]
%    \end{center}
    \caption{Approximate syntactic tree produced from the annotations in Figure~\ref{fig:chunking-spans}}
    \label{fig:chunking-tree}
  \end{figure*}


  The syntactic detail is not complete but chunking and constructing a tree this
  way is reasonably accurate and reliable for many purposes and much faster
  (especially for large documents over a web service) than full parsing.  (The
  verb chunker's annotations also contain features indicating tense, voice,
  etc., which will be incorporated into the SynAF output in the improved version
  of this service.)
%%
\item The \emph{annie-rdf} service runs ANNIE, then analyses ANNIE's annotations
  by type and features and generates RDF representing the recognized entities as
  instances according to the
  PROTON\footnote{\url{http://proton.semanticweb.org/}} \cite{Terziev2005}
  ontology, and returns an RDF-XML document.
\end{itemize}
%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Reference client}
%%%%%
We also provide a GUI Java client, supplied as a ZIP file with the necessary
libraries, so the user needs only a Java 5 runtime environment (JRE).  This
client uses the \emph{processWithParams} method and sends the \url{file://} URL,
and user-selected encoding along with the content of the selected local file.
The user selects the service from the list of endpoint URLs included with the
client, but can also type in a URL if he is aware of a service that has been
added since the client software was issued.  Figure~\ref{fig:client-controls}
shows this client's main panel used for sending files to the services, and
Figure~\ref{fig:client-output} shows the output panel, which allows the user to
inspect the output and save it to a local file.
%%
\begin{figure*}[tbp]
\begin{center}
\includegraphics[width=0.95\textwidth]{client-ss-controls.png}
\end{center}
\caption{Main panel of the GUI client}
\label{fig:client-controls}
\end{figure*}
%%
\begin{figure*}[tbp]
\begin{center}
\includegraphics[width=0.95\textwidth]{client-ss-output.png}
\end{center}
\caption{Output panel of the GUI client}
\label{fig:client-output}
\end{figure*}


Of course, developers can also use the services' WSDL files to produce their own
clients for users' direct use or embedment in other software.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Conclusion and future work}
\label{sec:conclusion}
%%%%%
The services described here have been proposed as contributions to CLARIN's
European Demonstrator.  We also plan to deploy services with MAF output for some
other European languages (probably a selection from Bulgarian, Dutch, German,
and Spanish) in the near future, based on the resources we have available, and
are open to suggestions for others, especially if suitable language resources
and processing tools are available to be shared with us and suitable for
integration with GATE.
%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section*{Acknowledgements}
%%%%% 
This research is partially supported by the European Union's Seventh Framework
Program project CLARIN (FP7-212230).
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\bibliographystyle{lrec2006}
\bibliography{../../big}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\end{document}
