de.aitools.ie.decomposition.tokenizer
Class WhitespaceTokenizer

java.lang.Object
  de.aitools.ie.decomposition.tokenizer.Tokenizer
      de.aitools.ie.decomposition.tokenizer.WhitespaceTokenizer

All Implemented Interfaces:

java.util.Iterator<java.lang.String>

public class WhitespaceTokenizer
extends Tokenizer

A whitespace tokenizer. Non whitespace sequences are identified as tokens.

Author:

[email protected]

Constructor Summary

WhitespaceTokenizer()

WhitespaceTokenizer(java.lang.String text)

Method Summary

boolean	hasNext()
java.lang.String	next()
void	setText(java.lang.String text)

Methods inherited from class de.aitools.ie.decomposition.tokenizer.Tokenizer

nextAsDeepCopy, remove

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

WhitespaceTokenizer

public WhitespaceTokenizer()

WhitespaceTokenizer

public WhitespaceTokenizer(java.lang.String text)

Method Detail

hasNext

public boolean hasNext()

next

public java.lang.String next()

setText

public void setText(java.lang.String text)

Specified by:

setText in class Tokenizer