de.aitools.dm.clusterlabeling.common
Interface Tokenizer

All Known Implementing Classes:

SimpleTokenizer

public interface Tokenizer

Version:

$Id: Tokenizer.java,v 1.1 2011/11/15 10:51:25 hoppe Exp $

Author:

dennis.hoppe(/\t)uni-weimar.de

Method Summary

java.util.Set<java.lang.String>	filter(java.util.Set<java.lang.String> text, java.lang.String filter)
java.util.Set<java.lang.String>	tokenize(java.lang.String text)

Method Detail

tokenize

java.util.Set<java.lang.String> tokenize(java.lang.String text)

filter

java.util.Set<java.lang.String> filter(java.util.Set<java.lang.String> text,
                                       java.lang.String filter)