%% Using Java code in JAPE rules
\section{Introduction}

The RHS of a JAPE rule can consist of any Java code. This is useful
for a variety of things such as removing temporary
annotations, percolating and manipulating features from previous
annotations, adding features to the document, etc.

The first rule below shows a rule which matches a first person name,
e.g. ``Fred'', and adds a gender feature depending on the value of the
minorType from the gazetteer list in which the name was found. We
first get the bindings associated with the person label (i.e. the
Lookup annotation). We then create a new annotation called ``personAnn''
which contains this annotation, and create a new FeatureMap to enable
us to add features. Then we get the minorType features (and its value)
from the personAnn annotation (in this case, the feature will be
``gender'' and the value will be ``male''), and add this value to a new
feature called ``gender''. We create another feature "rule" with value
``FirstName''. Finally, we add all the features to a new annotation
``FirstPerson'' which attaches to the same nodes as the original
``person'' binding.

\begin{verbatim}
Rule: FirstName
  
 (  
  {Lookup.majorType == person_first}  
 ):person  
 -->  
 {  
 gate.AnnotationSet person = (gate.AnnotationSet)bindings.get("person");  
 gate.Annotation personAnn = (gate.Annotation)person.iterator().next();  
 gate.FeatureMap features = Factory.newFeatureMap();  
 features.put("gender", personAnn.getFeatures().get("minorType"));  
 features.put("rule", "FirstName");  
 outputAS.add(person.firstNode(), person.lastNode(), "FirstPerson",  
 features);  
 } 
\end{verbatim}

Note that inputAS and outputAS represent the input and output
annotation set. Normally, these would be the same (by default when
using ANNIE, these will be the ``Default'' annotation set).  Since the
user is at liberty to change the input and output annotation sets in
the parameters of the JAPE transducer at runtime, it cannot be
guaranteed that the input and output annotation sets will be the same,
and therefore we must specify the annotation set to which we are referring.

The next rule (contained in a subsequent grammar phase) makes use of
annotations produced by the first rule described above. Instead of
percolating the minorType from the annotation produced by the
gazetteer lookup, this time it percolates the feature from the
annotation produced by the previous grammar rule. So here it gets the
``gender'' feature value from the ``FirstPerson'' annotation, and adds it
to a new feature (again called ``gender'' for convenience), which is
added to the new annotation (in outputAS) ``TempPerson''. At the end of
this rule, the existing input annotations (from inputAS) are removed
because they are no longer needed. Note that in the previous rule, the
existing annotations were not removed, because it is possible they
might be needed later on in another grammar phase.

\begin{verbatim}
 Rule: GazPersonFirst  
 (  
  {FirstPerson}  
 )  
 :person  
 -->  
 {  
 gate.AnnotationSet person = (gate.AnnotationSet)bindings.get("person");  
 gate.Annotation personAnn = (gate.Annotation)person.iterator().next();  
 gate.FeatureMap features = Factory.newFeatureMap();  
  
 features.put("gender", personAnn.getFeatures().get("gender"));  
 features.put("rule", "GazPersonFirst");  
 outputAS.add(person.firstNode(), person.lastNode(), "TempPerson",  
 features);  
 inputAS.removeAll(person);  
 } 
\end{verbatim}

\subsection{A more complex example}

The example below is more complicated, because both the title and the
first name (if present) may have a gender feature. There is a
possibility of conflict since some first names are ambiguous, or women
are given male names (e.g. Charlie). Some titles are also ambiguous,
such as ``Dr'', in which case they are not marked with a gender
feature. We therefore take the gender of the title in preference to
the gender of the first name, if it is present. So, on the RHS, we
first look for the gender of the title by getting all Title
annotations which have a gender feature attached. If a gender feature
is present, we add the value of this feature to a new gender feature
on the Person annotation we are going to create. If no gender feature
is present, we look for the gender of the first name by getting all
firstPerson annotations which have a gender feature attached, and adding
the value of this feature to a new gender feature on the Person
annotation we are going to create. If there is no firstPerson
annotation and the title has no gender information, then we simply
create the Person annotation with no gender feature.

\begin{verbatim}
Rule:	PersonTitle
Priority: 35
/* allows Mr. Jones, Mr Fred Jones etc. */
 
(
 (TITLE)
(FIRSTNAME | FIRSTNAMEAMBIG | INITIALS2)*
 (PREFIX)?
 {Upper}
 ({Upper})?
 (PERSONENDING)?
)
:person -->
 {
gate.FeatureMap features = Factory.newFeatureMap();
gate.AnnotationSet personSet = (gate.AnnotationSet)bindings.get("person");
  
// get all Title annotations that have a gender feature 
 HashSet fNames = new HashSet();
    fNames.add("gender");
    gate.AnnotationSet personTitle = personSet.get("Title", fNames);

// if the gender feature exists
 if (personTitle != null && personTitle.size()>0)
{
  gate.Annotation personAnn = (gate.Annotation)personTitle.iterator().next();
  features.put("gender", personAnn.getFeatures().get("gender"));
}
else
{
  // get all firstPerson annotations that have a gender feature
    gate.AnnotationSet firstPerson = personSet.get("FirstPerson", fNames);

  if (firstPerson != null && firstPerson.size()>0)
  // create a new gender feature and add the value from firstPerson
 {
  gate.Annotation personAnn = (gate.Annotation)firstPerson.iterator().next();
  features.put("gender", personAnn.getFeatures().get("gender"));
 }
}
  // create some other features
  features.put("kind", "personName");
  features.put("rule", "PersonTitle");
  // creat a Person annotation and add the features we've created
outputAS.add(personSet.firstNode(), personSet.lastNode(), "TempPerson",
features);
}
\end{verbatim}


\section{Adding a feature to the document}

This is useful when using conditional controllers, where we only want to fire
a particular resource under certain conditions. We first test the
document to see whether it fulfils these conditions or not, and attach
a feature to the document accordingly. 

In the example below, we test whether the document contains an 
annotation of type ``message''. In emails, there is often an annotation
of this type (produced by the document format analysis when the
document is loaded in GATE). Note that annotations produced by
document format analysis are placed automatically in the ``Original
markups'' annotation set, so we must ensure that when running the
processing resource containing this grammar that we specify the
Original markups set as the input annotation set. It does not matter
what we specify as the output annotation set, because the annotation
we produce is going to be attached to the document and not to an
output annotation set. In the example, if an annotation of type
``message'' is found, we add the feature "genre" with value "email" to
the document.

\begin{verbatim}
 Rule: Email  
 Priority: 150  
  
 (  
  {message}  
 )  
 -->  
 {  
 doc.getFeatures().put("genre", "email");  
 } 
\end{verbatim}


\section{Finding the Tokens of a matched annotation}

In this section we will demonstrate how by using Java on the
right-hand side one can find all Token annotations that are
covered by a matched annotation, e.g., a Person or an
Organization. This is useful if one wants to transfer some
information from the matched annotations to the tokens. For
example, to add to the Tokens a feature indicating whether or not
they are covered by a named entity annotation deduced by the
rule-based system. This feature can then be given as a feature to
a learning PR, e.g. the HMM. Similarly, one can add a feature to
all tokens saying which rule in the rule based system did the
match, the idea being that some rules might be more reliable than
others. Finally, yet another useful feature might be the length of
the coreference chain in which the matched entity is involved, if
such exists. 

The example below is one of the pre-processing JAPE grammars used
by the HMM application. To inspect all JAPE grammars, see the
muse/applications/hmm directory in the distribution.

\begin{verbatim}
Phase:  NEInfo

Input: Token Organization Location Person

Options: control = appelt

Rule:   NEInfo

Priority:100

({Organization} | {Person} | {Location}):entity
-->
{
  //get the annotation set
  gate.AnnotationSet annSet = ((gate.AnnotationSet)bindings.get("entity"));

  //get the only annotation from the set
  gate.Annotation entityAnn = (gate.Annotation)annSet.iterator().next();

  gate.AnnotationSet tokenAS = inputAS.get("Token",
        entityAnn.getStartNode().getOffset(),
        entityAnn.getEndNode().getOffset());
  List tokens = new ArrayList(tokenAS);
  //if no tokens to match, do nothing
  if (tokens.isEmpty())
     return;
  Collections.sort(tokens, new gate.util.OffsetComparator());

  gate.Annotation curToken=null;
  for (int i=0; i < tokens.size(); i++) {
    curToken = (gate.Annotation) tokens.get(i);
    String ruleInfo = (String) entityAnn.getFeatures().get("rule1");
    String NMRuleInfo = (String) entityAnn.getFeatures().get("NMRule");
    if ( ruleInfo != null) {
      curToken.getFeatures().put("rule_NE_kind", entityAnn.getType());
      curToken.getFeatures().put("NE_rule_id", ruleInfo);
    }
    else if (NMRuleInfo != null) {
      curToken.getFeatures().put("rule_NE_kind", entityAnn.getType());
      curToken.getFeatures().put("NE_rule_id", "orthomatcher");
    }
    else {
      curToken.getFeatures().put("rule_NE_kind", "None");
      curToken.getFeatures().put("NE_rule_id", "None");
    }
    List matchesList = (List) entityAnn.getFeatures().get("matches");
    if (matchesList != null) {
      if (matchesList.size() == 2)
        curToken.getFeatures().put("coref_chain_length", "2");
      else if (matchesList.size() > 2 && matchesList.size() < 5)
        curToken.getFeatures().put("coref_chain_length", "3-4");
    else
        curToken.getFeatures().put("coref_chain_length", "5-more");
    }
    else
      curToken.getFeatures().put("coref_chain_length", "0");
  }//for

}

Rule:   TokenNEInfo
Priority:10
({Token}):entity
-->
{
  //get the annotation set
  gate.AnnotationSet annSet = ((gate.AnnotationSet)bindings.get("entity"));

  //get the only annotation from the set
  gate.Annotation entityAnn = (gate.Annotation)annSet.iterator().next();

  entityAnn.getFeatures().put("rule_NE_kind", "None");
  entityAnn.getFeatures().put("NE_rule_id", "None");
  entityAnn.getFeatures().put("coref_chain_length", "0");
}

\end{verbatim}
