Package org.owasp.dependencycheck.data.lucene

Class TokenPairConcatenatingFilter

  • All Implemented Interfaces:
    java.io.Closeable, java.lang.AutoCloseable, org.apache.lucene.util.Unwrappable<org.apache.lucene.analysis.TokenStream>
    @NotThreadSafe
public final class TokenPairConcatenatingFilter
extends org.apache.lucene.analysis.TokenFilter

    Takes a TokenStream and adds additional tokens by concatenating pairs of words.

    Example: "Spring Framework Core" -> "Spring SpringFramework Framework FrameworkCore Core".

    Author:
    Jeremy Long

    • Nested Class Summary

      • Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource

        org.apache.lucene.util.AttributeSource.State

    • Field Summary

      • Fields inherited from class org.apache.lucene.analysis.TokenFilter

        input

      • Fields inherited from class org.apache.lucene.analysis.TokenStream

        DEFAULT_TOKEN_ATTRIBUTE_FACTORY

    • Constructor Summary

      Constructors 
      Constructor Description
      TokenPairConcatenatingFilter​(org.apache.lucene.analysis.TokenStream stream)
      Constructs a new TokenPairConcatenatingFilter.

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void clear()
      Resets the filter.
      boolean equals​(java.lang.Object obj)
      int hashCode()
      boolean incrementToken()
      Increments the underlying TokenStream and sets CharTermAttributes to construct an expanded set of tokens by concatenating tokens with the previous token.

      • Methods inherited from class org.apache.lucene.analysis.TokenFilter

        close, end, reset, unwrap

      • Methods inherited from class org.apache.lucene.util.AttributeSource

        addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, endAttributes, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, reflectAsString, reflectWith, removeAllAttributes, restoreState, toString

      • Methods inherited from class java.lang.Object

        clone, finalize, getClass, notify, notifyAll, wait, wait, wait

    • Constructor Detail

      • TokenPairConcatenatingFilter

        public TokenPairConcatenatingFilter​(org.apache.lucene.analysis.TokenStream stream)
        Constructs a new TokenPairConcatenatingFilter.
        Parameters:
        stream - the TokenStream that this filter will process

    • Method Detail

      • incrementToken

        public boolean incrementToken()
                       throws java.io.IOException
        Increments the underlying TokenStream and sets CharTermAttributes to construct an expanded set of tokens by concatenating tokens with the previous token.
        Specified by:
        incrementToken in class org.apache.lucene.analysis.TokenStream
        Returns:
        whether or not we have hit the end of the TokenStream
        Throws:
        java.io.IOException - is thrown when an IOException occurs

      • clear

        public void clear()
           throws java.io.IOException
        Resets the filter. This must be manually called between searching and indexing. Unable to rely on `reset` as it appears to be called between terms.
        Throws:
        java.io.IOException - thrown if there is an error reseting the tokenizer

      • hashCode

        public int hashCode()
        Overrides:
        hashCode in class org.apache.lucene.util.AttributeSource

      • equals

        public boolean equals​(java.lang.Object obj)
        Overrides:
        equals in class org.apache.lucene.util.AttributeSource