weka.classifiers.meta

Class MultiSearch

java.lang.Object

weka.classifiers.AbstractClassifier

weka.classifiers.SingleClassifierEnhancer

weka.classifiers.RandomizableSingleClassifierEnhancer

weka.classifiers.meta.MultiSearch

All Implemented Interfaces:

Serializable, Cloneable, weka.classifiers.Classifier, MultiSearchCapable, TraceableOptimizer, weka.core.AdditionalMeasureProducer, weka.core.BatchPredictor, weka.core.CapabilitiesHandler, weka.core.CapabilitiesIgnorer, weka.core.CommandlineRunnable, weka.core.OptionHandler, weka.core.Randomizable, weka.core.RevisionHandler, weka.core.Summarizable

public class MultiSearch
extends weka.classifiers.RandomizableSingleClassifierEnhancer
implements MultiSearchCapable, weka.core.AdditionalMeasureProducer, weka.core.Summarizable, TraceableOptimizer

Performs a search of an arbitrary number of parameters of a classifier and chooses the best pair found for the actual filtering and training.
The default MultiSearch is using the following Classifier setup:
LinearRegression, searching for the "Ridge"
The properties being explored are totally up to the user.

E.g., if you have a FilteredClassifier selected as base classifier, sporting a PLSFilter and you want to explore the number of PLS components, then your property will be made up of the following components:
- filter: referring to the FilteredClassifier's property (= PLSFilter)
- numComponents: the actual property of the PLSFilter that we want to modify
And assembled, the property looks like this:
filter.numComponents


The best classifier setup can be accessed after the buildClassifier call via the getBestClassifier method.

The trace of setups evaluated can be accessed after the buildClassifier call as well, using the following methods:
- getTrace()
- getTraceSize()
- getTraceValue(int)
- getTraceFolds(int)
- getTraceClassifierAsCli(int)
- getTraceParameterSettings(int)

Using the weka.core.setupgenerator.ParameterGroup parameter, it is possible to group dependent parameters. In this case, all top-level parameters must be of type weka.core.setupgenerator.ParameterGroup.

Valid options are:

 -E <CC|MCC|RMSE|RRSE|MAE|RAE|COMB|ACC|KAP|PREC|WPREC|REC|WREC|AUC|WAUC|PRC|WPRC|FM|WFM|TPR|TNR|FPR|FNR>
  Determines the parameter used for evaluation:
  CC = Correlation coefficient
  MCC = Matthews correlation coefficient
  RMSE = Root mean squared error
  RRSE = Root relative squared error
  MAE = Mean absolute error
  RAE = Root absolute error
  COMB = Combined = (1-abs(CC)) + RRSE + RAE
  ACC = Accuracy
  KAP = Kappa
  PREC = Precision (per class)
  WPREC = Weighted precision
  REC = Recall (per class)
  WREC = Weighted recall
  AUC = Area under ROC (per class)
  WAUC = Weighted area under ROC
  PRC = Area under PRC (per class)
  WPRC = Weighted area under PRC
  FM = F-Measure (per class)
  WFM = Weighted F-Measure
  TPR = True positive rate (per class)
  TNR = True negative rate (per class)
  FPR = False positive rate (per class)
  FNR = False negative rate (per class)
  (default: CC)

 -class-label "<1-based index>"
  The class label index to retrieve the metric for (if applicable).
 

 -search "<classname options>"
  A property search setup.
 

 -algorithm "<classname options>"
  A search algorithm.
 

 -log-file <filename>
  The log file to log the messages to.
  (default: none)

 -S <num>
  Random number seed.
  (default 1)

 -W
  Full name of base classifier.
  (default: weka.classifiers.functions.LinearRegression)

 -output-debug-info
  If set, classifier is run in debug mode and
  may output additional info to the console

 -do-not-check-capabilities
  If set, classifier capabilities are not checked before classifier is built
  (use with caution).

 -num-decimal-places
  The number of decimal places for the output of numbers in the model (default 2).

 -batch-size
  The desired batch size for batch prediction  (default 100).

 
 Options specific to classifier weka.classifiers.functions.LinearRegression:
 

 -S <number of selection method>
  Set the attribute selection method to use. 1 = None, 2 = Greedy.
  (default 0 = M5' method)

 -C
  Do not try to eliminate colinear attributes.
 

 -S <number of selection method>
  Set the attribute selection method to use. 1 = None, 2 = Greedy.
  (default 0 = M5' method)

 -R <double>
  Set ridge parameter (default 1.0e-8).
 

 -minimal
  Conserve memory, don't keep dataset header and means/stdevs.
  Model cannot be printed out if this option is enabled. (default: keep data)

 -additional-stats
  Output additional statistics.

 -output-debug-info
  If set, classifier is run in debug mode and
  may output additional info to the console

 -do-not-check-capabilities
  If set, classifier capabilities are not checked before classifier is built
  (use with caution).

 -num-decimal-places
  The number of decimal places for the output of numbers in the model (default 4).

 -batch-size
  The desired batch size for batch prediction  (default 100).

Author:

fracpete (fracpete at waikato dot ac dot nz)

See Also:

Serialized Form

Field Summary

Fields inherited from class weka.classifiers.AbstractClassifier

BATCH_SIZE_DEFAULT, NUM_DECIMAL_PLACES_DEFAULT

Constructor Summary

Constructors

Constructor and Description
MultiSearch() the default constructor.

Method Summary

Modifier and Type	Method and Description
String	algorithmTipText() Returns the tip text for this property.
void	buildClassifier(weka.core.Instances data) builds the classifier.
String	classLabelTipText() Returns the tip text for this property.
AbstractSearch	defaultAlgorithm() Creates the default search algorithm.
double[]	distributionForInstance(weka.core.Instance instance) Returns the distribution for the given instance.
Enumeration	enumerateMeasures() Returns an enumeration of the measure names.
String	evaluationTipText() Returns the tip text for this property.
AbstractSearch	getAlgorithm() Returns the search algorithm.
weka.classifiers.Classifier	getBestClassifier() returns the best Classifier setup.
Point<Object>	getBestCoordinates() returns the points that were found to work best.
Point<Object>	getBestValues() returns the parameter values that were found to work best.
weka.core.Capabilities	getCapabilities() Returns default capabilities of the classifier.
String	getClassLabel() Returns the class label to retrieve the metrics for (if applicable).
int	getClassLabelIndex(int upper) Returns the integer index.
String	getCommandline(Object obj) Returns the commandline of the given object.
weka.core.SelectedTag	getEvaluation() Gets the criterion used for evaluating the classifier performance.
AbstractEvaluationFactory	getFactory() Returns the factory instance.
SetupGenerator	getGenerator() Returns the setup generator.
File	getLogFile() Gets current log file.
double	getMeasure(String measureName) Returns the value of the named measure.
AbstractEvaluationMetrics	getMetrics() Returns the evaluation metrics.
weka.core.Tag[]	getMetricsTags() Returns the underlying tags.
String[]	getOptions() returns the options of the current setup.
String	getRevision() Returns the revision string.
AbstractParameter[]	getSearchParameters() Returns the search parameters.
List<Map.Entry<Integer,Performance>>	getTrace() Returns the full trace.
String	getTraceClassifierAsCli(int index) Returns the CLI string of a given item in the trace.
Integer	getTraceFolds(int index) Returns the folds of a given item in the trace.
List<Map.Entry<String,Object>>	getTraceParameterSettings(int index) Returns the parameter settings in structured way
int	getTraceSize() Returns the size of m_Trace, which is technically the amount of setups that where tested in order to find the best.
Double	getTraceValue(int index) Returns the performance score of a given item in the trace.
String	globalInfo() Returns a string describing classifier.
Enumeration	listOptions() Gets an enumeration describing the available options.
void	log(String message) prints the specified message to stdout if debug is on and can also dump the message to a log file.
void	log(String message, boolean onlyLog) prints the specified message to stdout if debug is on and can also dump the message to a log file.
String	logFileTipText() Returns the tip text for this property.
void	logPerformances(Space space, Vector<Performance> performances) aligns all performances in the space and prints those tables to the log file.
String	logPerformances(Space space, Vector<Performance> performances, weka.core.Tag type) generates a table string for all the performances in the space and returns that.
static void	main(String[] args) Main method for running this classifier from commandline.
String	searchParametersTipText() Returns the tip text for this property.
void	setAlgorithm(AbstractSearch value) Sets the search algorithm.
void	setClassifier(weka.classifiers.Classifier newClassifier) Set the base learner.
void	setClassLabel(String value) Sets the class label to retrieve the metrics for (if applicable).
void	setEvaluation(weka.core.SelectedTag value) Sets the criterion to use for evaluating the classifier performance.
void	setLogFile(File value) Sets the log file to use.
void	setOptions(String[] options) Parses the options for this object.
void	setSearchParameters(AbstractParameter[] value) Sets the search parameters.
String	toString() returns a string representation of the classifier.
String	toSummaryString() Returns a string that summarizes the object.

Methods inherited from class weka.classifiers.RandomizableSingleClassifierEnhancer

getSeed, seedTipText, setSeed

Methods inherited from class weka.classifiers.SingleClassifierEnhancer

classifierTipText, getClassifier, postExecution, preExecution

Methods inherited from class weka.classifiers.AbstractClassifier

batchSizeTipText, classifyInstance, debugTipText, distributionsForInstances, doNotCheckCapabilitiesTipText, forName, getBatchSize, getDebug, getDoNotCheckCapabilities, getNumDecimalPlaces, implementsMoreEfficientBatchPrediction, makeCopies, makeCopy, numDecimalPlacesTipText, run, runClassifier, setBatchSize, setDebug, setDoNotCheckCapabilities, setNumDecimalPlaces

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface weka.classifiers.meta.multisearch.MultiSearchCapable

getClassifier, getSeed

Methods inherited from interface weka.classifiers.Classifier

classifyInstance

Methods inherited from interface weka.core.OptionHandler

makeCopy

Constructor Detail

MultiSearch

public MultiSearch()

the default constructor.

Method Detail

globalInfo

public String globalInfo()

Returns a string describing classifier.

Returns:

a description suitable for displaying in the explorer/experimenter gui

listOptions

public Enumeration listOptions()

Gets an enumeration describing the available options.

Specified by:

listOptions in interface weka.core.OptionHandler

Overrides:

listOptions in class weka.classifiers.RandomizableSingleClassifierEnhancer

Returns:

an enumeration of all the available options.

getOptions

public String[] getOptions()

returns the options of the current setup.

Specified by:

getOptions in interface weka.core.OptionHandler

Overrides:

getOptions in class weka.classifiers.RandomizableSingleClassifierEnhancer

Returns:

the current options

setOptions

public void setOptions(String[] options)
                throws Exception

Parses the options for this object.

Specified by:

setOptions in interface weka.core.OptionHandler

Overrides:

setOptions in class weka.classifiers.RandomizableSingleClassifierEnhancer

Parameters:

options - the options to use

Throws:

Exception - if setting of options fails

setClassifier

public void setClassifier(weka.classifiers.Classifier newClassifier)

Set the base learner.

Overrides:

setClassifier in class weka.classifiers.SingleClassifierEnhancer

Parameters:

newClassifier - the classifier to use.

searchParametersTipText

public String searchParametersTipText()

Returns the tip text for this property.

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

setSearchParameters

public void setSearchParameters(AbstractParameter[] value)

Sets the search parameters.

Parameters:

value - the parameters

getSearchParameters

public AbstractParameter[] getSearchParameters()

Returns the search parameters.

Returns:

the parameters

algorithmTipText

public String algorithmTipText()

Returns the tip text for this property.

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

setAlgorithm

public void setAlgorithm(AbstractSearch value)

Sets the search algorithm.

Parameters:

value - the algorithm

getAlgorithm

public AbstractSearch getAlgorithm()

Returns the search algorithm.

Specified by:

getAlgorithm in interface MultiSearchCapable

Returns:

the algorithm

classLabelTipText

public String classLabelTipText()

Returns the tip text for this property.

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

setClassLabel

public void setClassLabel(String value)

Sets the class label to retrieve the metrics for (if applicable).

Parameters:

value - the class lable index (1-based)

getClassLabel

public String getClassLabel()

Returns the class label to retrieve the metrics for (if applicable).

Returns:

the class label index (1-based)

getClassLabelIndex

public int getClassLabelIndex(int upper)

Returns the integer index.

Specified by:

getClassLabelIndex in interface MultiSearchCapable

Parameters:

upper - the maximum to use

Returns:

the index (0-based)

defaultAlgorithm

public AbstractSearch defaultAlgorithm()

Creates the default search algorithm.

Returns:

the algorithm

evaluationTipText

public String evaluationTipText()

Returns the tip text for this property.

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getMetricsTags

public weka.core.Tag[] getMetricsTags()

Returns the underlying tags.

Returns:

the tags

setEvaluation

public void setEvaluation(weka.core.SelectedTag value)

Sets the criterion to use for evaluating the classifier performance.

Parameters:

value - the evaluation criterion

getEvaluation

public weka.core.SelectedTag getEvaluation()

Gets the criterion used for evaluating the classifier performance.

Specified by:

getEvaluation in interface MultiSearchCapable

Returns:

the current evaluation criterion.

logFileTipText

public String logFileTipText()

Returns the tip text for this property.

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getLogFile

public File getLogFile()

Gets current log file.

Returns:

the log file.

setLogFile

public void setLogFile(File value)

Sets the log file to use.

Parameters:

value - the log file.

getBestClassifier

public weka.classifiers.Classifier getBestClassifier()

returns the best Classifier setup.

Specified by:

getBestClassifier in interface MultiSearchCapable

Returns:

the best Classifier setup

getGenerator

public SetupGenerator getGenerator()

Returns the setup generator.

Specified by:

getGenerator in interface MultiSearchCapable

Returns:

the generator

enumerateMeasures

public Enumeration enumerateMeasures()

Returns an enumeration of the measure names.

Specified by:

enumerateMeasures in interface weka.core.AdditionalMeasureProducer

Returns:

an enumeration of the measure names

getMeasure

public double getMeasure(String measureName)

Returns the value of the named measure.

Specified by:

getMeasure in interface weka.core.AdditionalMeasureProducer

Parameters:

measureName - the name of the measure to query for its value

Returns:

the value of the named measure

getFactory

public AbstractEvaluationFactory getFactory()

Returns the factory instance.

Specified by:

getFactory in interface MultiSearchCapable

Returns:

the factory

getMetrics

public AbstractEvaluationMetrics getMetrics()

Returns the evaluation metrics.

Specified by:

getMetrics in interface MultiSearchCapable

Returns:

the metrics

getBestValues

public Point<Object> getBestValues()

returns the parameter values that were found to work best.

Specified by:

getBestValues in interface MultiSearchCapable

Returns:

the best parameter combination

getBestCoordinates

public Point<Object> getBestCoordinates()

returns the points that were found to work best.

Specified by:

getBestCoordinates in interface MultiSearchCapable

Returns:

the best points

getCapabilities

public weka.core.Capabilities getCapabilities()

Returns default capabilities of the classifier.

Specified by:

getCapabilities in interface weka.classifiers.Classifier

Specified by:

getCapabilities in interface weka.core.CapabilitiesHandler

Overrides:

getCapabilities in class weka.classifiers.SingleClassifierEnhancer

Returns:

the capabilities of this classifier

getCommandline

public String getCommandline(Object obj)

Returns the commandline of the given object.

Specified by:

getCommandline in interface MultiSearchCapable

Parameters:

obj - the object to create the commandline for

Returns:

the commandline

log

public void log(String message)

prints the specified message to stdout if debug is on and can also dump the message to a log file.

Specified by:

log in interface MultiSearchCapable

Parameters:

message - the message to print or store in a log file

log

public void log(String message,
                boolean onlyLog)

prints the specified message to stdout if debug is on and can also dump the message to a log file.

Specified by:

log in interface MultiSearchCapable

Parameters:

message - the message to print or store in a log file

onlyLog - if true the message will only be put into the log file but not to stdout

logPerformances

public String logPerformances(Space space,
                              Vector<Performance> performances,
                              weka.core.Tag type)

generates a table string for all the performances in the space and returns that.

Specified by:

logPerformances in interface MultiSearchCapable

Parameters:

space - the current space to align the performances to

performances - the performances to align

type - the type of performance

Returns:

the table string

logPerformances

public void logPerformances(Space space,
                            Vector<Performance> performances)

aligns all performances in the space and prints those tables to the log file.

Specified by:

logPerformances in interface MultiSearchCapable

Parameters:

space - the current space to align the performances to

performances - the performances to align

getTraceSize

public int getTraceSize()

Returns the size of m_Trace, which is technically the amount of setups that where tested in order to find the best.

Specified by:

getTraceSize in interface TraceableOptimizer

Returns:

the number of setups tested

getTraceClassifierAsCli

public String getTraceClassifierAsCli(int index)

Returns the CLI string of a given item in the trace.

Specified by:

getTraceClassifierAsCli in interface TraceableOptimizer

Parameters:

index - the index of the trace item to obtain

Returns:

the commandline

getTraceValue

public Double getTraceValue(int index)

Returns the performance score of a given item in the trace.

Specified by:

getTraceValue in interface TraceableOptimizer

Parameters:

index - the index of the trace item to obtain

Returns:

the trace value

getTraceParameterSettings

public List<Map.Entry<String,Object>> getTraceParameterSettings(int index)

Returns the parameter settings in structured way

Specified by:

getTraceParameterSettings in interface TraceableOptimizer

Parameters:

index - the index of the trace item to obtain

Returns:

the parameter settings

getTraceFolds

public Integer getTraceFolds(int index)

Returns the folds of a given item in the trace.

Specified by:

getTraceFolds in interface TraceableOptimizer

Parameters:

index - the index of the trace item to obtain

Returns:

the number of folds

getTrace

public List<Map.Entry<Integer,Performance>> getTrace()

Returns the full trace.

Specified by:

getTrace in interface TraceableOptimizer

Returns:

the full trace

buildClassifier

public void buildClassifier(weka.core.Instances data)
                     throws Exception

builds the classifier.

Specified by:

buildClassifier in interface weka.classifiers.Classifier

Parameters:

data - the training instances

Throws:

Exception - if something goes wrong

distributionForInstance

public double[] distributionForInstance(weka.core.Instance instance)
                                 throws Exception

Returns the distribution for the given instance.

Specified by:

distributionForInstance in interface weka.classifiers.Classifier

Overrides:

distributionForInstance in class weka.classifiers.AbstractClassifier

Parameters:

instance - the test instance

Returns:

the distribution array

Throws:

Exception - if distribution can't be computed successfully

toString

public String toString()

returns a string representation of the classifier.

Overrides:

toString in class Object

Returns:

a string representation of the classifier

toSummaryString

public String toSummaryString()

Returns a string that summarizes the object.

Specified by:

toSummaryString in interface weka.core.Summarizable

Returns:

the object summarized as a string

getRevision

public String getRevision()

Returns the revision string.

Specified by:

getRevision in interface weka.core.RevisionHandler

Overrides:

getRevision in class weka.classifiers.AbstractClassifier

Returns:

the revision

main

public static void main(String[] args)

Main method for running this classifier from commandline.

Parameters:

args - the options