Uses of Package
gate.creole.tokeniser

Packages that use gate.creole.tokeniser
gate.creole.tokeniser   
 

Classes in gate.creole.tokeniser used by gate.creole.tokeniser
DFSMState
          Implements a state of the deterministic finite state machine of the tokeniser.
FSMState
          A state of the finite state machine that is the kernel tokeniser
SimpleTokeniser
          Implementation of a Unicode rule based tokeniser.
TokeniserException
          The top level exception for all the exceptions fired by the tokeniser
UnicodeType
          Used as an object wrapper that holds an Unicode type (the byte value of the static member of java.lang.Character).