smile.nlp.keyword

Interface CooccurrenceKeywords

public interface CooccurrenceKeywords

Keyword extraction from a single document using word co-occurrence statistical information. The algorithm was proposed by Y. Matsuo and M. Ishizuka. It consists of six steps:

1. Stem words by Porter algorithm and extract phrases based APRIORI algorithm (upto 4 words with frequency more than 3 times). Discard stop words.
2. Select the top frequent terms up to 30% of running terms.
3. Clustering frequent terms. Two terms are in the same cluster if either their Jensen-Shannon divergence or mutual information is above the threshold (0.95 * log 2, and log 2, respectively).
4. Calculate the expected co-occurrence probability
5. Calculate the refined χ2 values that removes the maximal term.
6. Output a given number of terms of largest refined χ2 values.

Method Summary

Modifier and Type	Method and Description
static NGram[]	of(java.lang.String text) Returns the top 10 keywords.
static NGram[]	of(java.lang.String text, int maxNumKeywords) Returns a given number of top keywords.

Method Detail

of

static NGram[] of(java.lang.String text)

Returns the top 10 keywords.

Parameters:

text - A single document.

Returns:

The top 10 keywords.

of

static NGram[] of(java.lang.String text,
                  int maxNumKeywords)

Returns a given number of top keywords.

Parameters:

text - A single document.

Returns:

The top keywords.