SoftTFIDFDictionary (SecondString 0.1 API)

java.lang.Object
- com.wcohen.ss.lookup.SoftTFIDFDictionary

All Implemented Interfaces:

FastLookup
```
public class SoftTFIDFDictionary
extends Object
implements FastLookup
```
Looks up nearly-matching strings in a dictionary, using SoftTFIDF distance. To use the dictionary, first load in string/value pairs using 'put'. Then 'freeze' the dictionary. After the dictionary is frozen, you can lookup values with lookup and getResult(i), getValue(i), etc.
For example:
```
 SoftTFIDFDictionary dict = new SoftTFIDFDictionary();
 dict.put("william cohen", "wcohen@cs.cmu.edu");
 dict.put("vitor del rocha carvalho", "vitor@cs.cmu.edu");
 ...
 dict.freeze();
 int n=dict.lookup("victor carvalho");
 for (int i=0; i
```

Field Summary

Fields
Modifier and Type Field and Description

protected double lookupTime

Fields
Modifier and Type	Field and Description
`protected double`	`lookupTime`

Constructor Summary

Constructors
Constructor and Description
`SoftTFIDFDictionary()`
`SoftTFIDFDictionary(Tokenizer tokenizer)`
`SoftTFIDFDictionary(Tokenizer tokenizer, double minTokenSimilarity)`
`SoftTFIDFDictionary(Tokenizer tokenizer, double minTokenSimilarity, int windowSize, int maxInvertedIndexSize)` Create a new SoftTFIDFDictionary.

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`void`	`freeze()` Make it impossible to add new values, but possible to perform lookups.
`double`	`getLookupTime()` Get the time used in performing the lookup
`int`	`getMaxInvertedIndexSize()`
`String`	`getResult(int i)` Get the i'th string found by the last lookup
`double`	`getScore(int i)` Get the score of the i'th string found by the last lookup
`Object`	`getValue(int i)` Get the value of the i'th string found by the last lookup
`int`	`getWindowSize(int w)`
`void`	`loadAliases(File file)` Load a file of identifiers, each of which has multiple aliases.
`int`	`lookup(double minScore, String toFind)` Lookup items SoftTFIDF-similar to the 'toFind' argument, and return the number of items found.
`static void`	`main(String[] argv)` Simple main for testing and experimentation
`void`	`put(String string, Object value)` Insert a string into the dictionary, and associate it with the given value.
`void`	`refreeze()`
`static SoftTFIDFDictionary`	`restore(File file)`
`void`	`saveAs(File file)`
`void`	`setMaxInvertedIndexSize(int m)` Set the maximum size of an inverted index that will be followed.
`void`	`setWindowSize(int w)` Set the 'windowSize' used for finding similar tokens.
`int`	`slowLookup(double minScore, String toFind)` Exactly like lookup, but works by exhaustively checking every stored string.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - lookupTime
```
protected double lookupTime
```
- Constructor Detail
  - SoftTFIDFDictionary
```
public SoftTFIDFDictionary()
```
  - SoftTFIDFDictionary
```
public SoftTFIDFDictionary(Tokenizer tokenizer)
```
  - SoftTFIDFDictionary
```
public SoftTFIDFDictionary(Tokenizer tokenizer,
                           double minTokenSimilarity)
```
  - SoftTFIDFDictionary
```
public SoftTFIDFDictionary(Tokenizer tokenizer,
                           double minTokenSimilarity,
                           int windowSize,
                           int maxInvertedIndexSize)
```
    Create a new SoftTFIDFDictionary. The distance is defined by a SoftTFIDF distance function where minTokenSimilarity is the minimum Jaro-Winkler distance between similar tokens, and the tokenizer defines the tokens considered.
- Method Detail
  - saveAs
```
public void saveAs(File file)
            throws IOException,
                   FileNotFoundException
```
    Throws:
    
    IOException
    
    FileNotFoundException
  - restore
```
public static SoftTFIDFDictionary restore(File file)
                                   throws IOException,
                                          FileNotFoundException
```
    Throws:
    
    IOException
    
    FileNotFoundException
  - setWindowSize
```
public void setWindowSize(int w)
```
    Set the 'windowSize' used for finding similar tokens. When finding tokens t2 that are similar to a given t1, the dictionary limits itself to tokens t3 that are within distance 'windowSize' of t1 on a sorted list of all tokens in the dictionary
  - getWindowSize
```
public int getWindowSize(int w)
```
  - setMaxInvertedIndexSize
```
public void setMaxInvertedIndexSize(int m)
```
    Set the maximum size of an inverted index that will be followed. If this is zero (the default) then any inverted index will be followed, even for very frequent tokens, if following it is justified by the upper bound algorithms.
  - getMaxInvertedIndexSize
```
public int getMaxInvertedIndexSize()
```
  - loadAliases
```
public void loadAliases(File file)
                 throws IOException,
                        FileNotFoundException
```
    Load a file of identifiers, each of which has multiple aliases. The dictionary constructed will map aliases to identifiers. Each line in the file is a list of tab-separated strings, the first of which is the identifier, the remainder of which are aliases.
    
    Throws:
    
    IOException
    
    FileNotFoundException
  - put
```
public void put(String string,
                Object value)
```
    Insert a string into the dictionary, and associate it with the given value.
  - refreeze
```
public void refreeze()
```
  - freeze
```
public void freeze()
```
    Make it impossible to add new values, but possible to perform lookups.
  - slowLookup
```
public int slowLookup(double minScore,
                      String toFind)
```
    Exactly like lookup, but works by exhaustively checking every stored string.
  - lookup
```
public int lookup(double minScore,
                  String toFind)
```
    Lookup items SoftTFIDF-similar to the 'toFind' argument, and return the number of items found. The looked-up items must have a similarity score greater than minScore.
    
    Specified by:
    
    lookup in interface FastLookup
  - getResult
```
public String getResult(int i)
```
    Get the i'th string found by the last lookup
    
    Specified by:
    
    getResult in interface FastLookup
  - getValue
```
public Object getValue(int i)
```
    Get the value of the i'th string found by the last lookup
    
    Specified by:
    
    getValue in interface FastLookup
  - getScore
```
public double getScore(int i)
```
    Get the score of the i'th string found by the last lookup
    
    Specified by:
    
    getScore in interface FastLookup
  - getLookupTime
```
public double getLookupTime()
```
    Get the time used in performing the lookup
  - main
```
public static void main(String[] argv)
                 throws IOException,
                        FileNotFoundException,
                        NumberFormatException,
                        ClassNotFoundException
```
    Simple main for testing and experimentation
    
    Throws:
    
    IOException
    
    FileNotFoundException
    
    NumberFormatException
    
    ClassNotFoundException

Class SoftTFIDFDictionary

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

lookupTime

Constructor Detail

SoftTFIDFDictionary

SoftTFIDFDictionary

SoftTFIDFDictionary

SoftTFIDFDictionary

Method Detail

saveAs

restore

setWindowSize

getWindowSize

setMaxInvertedIndexSize

getMaxInvertedIndexSize

loadAliases

put

refreeze

freeze

slowLookup

lookup

getResult

getValue

getScore

getLookupTime

main