Uses of Class
de.aitools.ie.decomposition.tokenizer.Tokenizer

Packages that use Tokenizer

de.aitools.ie.decomposition.chunker
de.aitools.ie.decomposition.tokenizer
de.aitools.ie.decomposition.usage

Uses of Tokenizer in de.aitools.ie.decomposition.chunker

Methods in de.aitools.ie.decomposition.chunker with parameters of type Tokenizer

void	Chunker.setTokenizer(Tokenizer tokenizer)

Constructors in de.aitools.ie.decomposition.chunker with parameters of type Tokenizer

Chunker(Tokenizer tokenizer)

NGramChunker(Tokenizer tokenizer, int ngramSize)

SlidingWindow(Tokenizer tokenizer, int windowSize, int shiftSize)

Uses of Tokenizer in de.aitools.ie.decomposition.tokenizer

Subclasses of Tokenizer in de.aitools.ie.decomposition.tokenizer

class	CharClassTokenizer A character class tokenizer.
class	CharNGramTokenizer
class	EntityTokenizer
class	LocalizedTokenizer
class	SentenceTokenizer
class	WhitespaceTokenizer A whitespace tokenizer.
class	WordTokenizer

Uses of Tokenizer in de.aitools.ie.decomposition.usage

Fields in de.aitools.ie.decomposition.usage declared as Tokenizer

static Tokenizer[]

TokenizerPreview.tokenizerRegistry