net.java.sen.filter.stream
Class CompositeTokenFilter

java.lang.Object
  net.java.sen.filter.stream.CompositeTokenFilter

All Implemented Interfaces:

StreamFilter

public class CompositeTokenFilter
extends Object
implements StreamFilter

A Filter that replaces multiple similar Tokens with a single composite Token

Constructor Summary

CompositeTokenFilter()

Method Summary

List<Token>	postProcess(List<Token> tokens) Post-processes analysed tokens
void	preProcess(Sentence sentence) Pre-processes a sentence
void	readRules(BufferedReader reader) Reads the rules to apply as space-delimited text

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CompositeTokenFilter

public CompositeTokenFilter()

Method Detail

readRules

public void readRules(BufferedReader reader)
               throws IOException

Reads the rules to apply as space-delimited text

Parameters:

reader - The reader from which to read the rules

Throws:

IOException

preProcess

public void preProcess(Sentence sentence)

Description copied from interface: StreamFilter

Pre-processes a sentence

Specified by:

preProcess in interface StreamFilter

Parameters:

sentence - The sentence be pre-processed

postProcess

public List<Token> postProcess(List<Token> tokens)

Description copied from interface: StreamFilter

Post-processes analysed tokens

Specified by:

postProcess in interface StreamFilter

Parameters:

tokens - The analysed tokens

Returns:

The post-processed tokens