de.aitools.ie.decomposition.tokenizer
Class WhitespaceTokenizer

java.lang.Object
  extended by de.aitools.ie.decomposition.tokenizer.Tokenizer
      extended by de.aitools.ie.decomposition.tokenizer.WhitespaceTokenizer
All Implemented Interfaces:
java.util.Iterator<java.lang.String>

public class WhitespaceTokenizer
extends Tokenizer

A whitespace tokenizer. Non whitespace sequences are identified as tokens.

Author:
martin.trenkmann@uni-weimar.de

Constructor Summary
WhitespaceTokenizer()
           
WhitespaceTokenizer(java.lang.String text)
           
 
Method Summary
 boolean hasNext()
           
 java.lang.String next()
           
 void setText(java.lang.String text)
           
 
Methods inherited from class de.aitools.ie.decomposition.tokenizer.Tokenizer
nextAsDeepCopy, remove
 
Methods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

WhitespaceTokenizer

public WhitespaceTokenizer()

WhitespaceTokenizer

public WhitespaceTokenizer(java.lang.String text)
Method Detail

hasNext

public boolean hasNext()

next

public java.lang.String next()

setText

public void setText(java.lang.String text)
Specified by:
setText in class Tokenizer