|
|||||||||||
PREV NEXT | FRAMES NO FRAMES |
Packages that use gate.creole.tokeniser | |
gate.creole.morph | |
gate.creole.tokeniser | |
gate.creole.tokeniser.chinesetokeniser |
Classes in gate.creole.tokeniser used by gate.creole.morph | |
DefaultTokeniser
A composed tokeniser containing a SimpleTokeniser and a
Transducer .
|
Classes in gate.creole.tokeniser used by gate.creole.tokeniser | |
DFSMState
Implements a state of the deterministic finite state machine of the tokeniser. |
|
FSMState
A state of the finite state machine that is the kernel tokeniser |
|
SimpleTokeniser
Implementation of a Unicode rule based tokeniser. |
|
TokeniserException
The top level exception for all the exceptions fired by the tokeniser |
|
UnicodeType
Used as an object wrapper that holds an Unicode type (the byte value of the static member of java.lang.Character). |
Classes in gate.creole.tokeniser used by gate.creole.tokeniser.chinesetokeniser | |
SimpleTokeniser
Implementation of a Unicode rule based tokeniser. |
|
|||||||||||
PREV NEXT | FRAMES NO FRAMES |