de.aitools.ie.decomposition.tokenizer
Class CharClassTokenizer

java.lang.Object
  de.aitools.ie.decomposition.tokenizer.Tokenizer
      de.aitools.ie.decomposition.tokenizer.CharClassTokenizer

All Implemented Interfaces:

java.util.Iterator<java.lang.String>

public class CharClassTokenizer
extends Tokenizer

A character class tokenizer. Sequences of the same character class are tokens.

Author:

martin.trenkmann@uni-weimar.de

Constructor Summary

CharClassTokenizer()

CharClassTokenizer(java.lang.String text)

Method Summary

boolean	hasNext()
java.lang.String	next()
void	setText(java.lang.String text)

Methods inherited from class de.aitools.ie.decomposition.tokenizer.Tokenizer

nextAsDeepCopy, remove

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CharClassTokenizer

public CharClassTokenizer()

CharClassTokenizer

public CharClassTokenizer(java.lang.String text)

Method Detail

hasNext

public boolean hasNext()

next

public java.lang.String next()

setText

public void setText(java.lang.String text)

Specified by:

setText in class Tokenizer