org.nd4j.linalg.lossfunctions.impl

Class LossMultiLabel

java.lang.Object

org.nd4j.autodiff.functions.DifferentialFunction

org.nd4j.linalg.lossfunctions.impl.LossMultiLabel

All Implemented Interfaces:

Serializable, ILossFunction

public class LossMultiLabel
extends DifferentialFunction
implements ILossFunction

Multi-Label-Loss Function, maybe more commonly known as BPMLL

This Loss function requires that the Labels are given as a multi-hot encoded vector. It doesn't require any special Activation method, i.e. the network output doesn't have to be in any specific range.

The loss is calculated based on the classification difference on labels that the examples has, and those that it doesn't have. Assume that each example has a set of labels, these labels are the positive set, the labels that do not belong to the example are in the negative set. This loss function trains the network to produce a higher value for labels that are in the positive set than those that are in the negative set.

For a more detailed explanation and the actual formulas, read the original paper by Zhang and Zhou. The implementation on scoreArray is based on equation 3, while computeGradient is based on equation 11. The main difference being that -(c_k - c_l) = (c_l - c_k) was used to simplify the calculations.

Min-Ling Zhang and Zhi-Hua Zhou, "Multilabel Neural Networks with Applications to Functional Genomics and Text Categorization," in IEEE Transactions on Knowledge and Data Engineering, vol. 18, no. 10, pp. 1338-1351, Oct. 2006. doi: 10.1109/TKDE.2006.162 *

Author:

Paul Dubs

See Also:

Serialized Form

Field Summary

Fields inherited from class org.nd4j.autodiff.functions.DifferentialFunction

dimensions, extraArgs, inPlace, sameDiff, scalarValue

Constructor Summary

Constructors

Constructor and Description
LossMultiLabel()

Method Summary

Modifier and Type	Method and Description
INDArray	computeGradient(INDArray labels, INDArray preOutput, IActivation activationFn, INDArray mask) Compute the gradient of the loss function with respect to the inputs: dL/dOutput
org.nd4j.linalg.primitives.Pair<Double,INDArray>	computeGradientAndScore(INDArray labels, INDArray preOutput, IActivation activationFn, INDArray mask, boolean average) Compute both the score (loss function value) and gradient.
double	computeScore(INDArray labels, INDArray preOutput, IActivation activationFn, INDArray mask, boolean average) Compute the score (loss function value) for the given inputs.
INDArray	computeScoreArray(INDArray labels, INDArray preOutput, IActivation activationFn, INDArray mask) Compute the score (loss function value) for each example individually.
List<SDVariable>	doDiff(List<SDVariable> f1) The actual implementation for automatic differentiation.
void	initFromOnnx(OnnxProto3.NodeProto node, SameDiff initWith, Map<String,OnnxProto3.AttributeProto> attributesForNode, OnnxProto3.GraphProto graph) Iniitialize the function from the given OnnxProto3.NodeProto
void	initFromTensorFlow(NodeDef nodeDef, SameDiff initWith, Map<String,AttrValue> attributesForNode, GraphDef graph) Initialize the function from the given NodeDef
String	name() The opName of this function
String	onnxName() The opName of this function in onnx
String	opName() The name of the op
Op.Type	opType() The type of the op
SDVariable[]	outputVariables() Return the output variables for this differential function.
SDVariable[]	outputVariables(String baseName) Return the output functions for this differential function.
INDArray	scoreArray(INDArray labels, INDArray preOutput, IActivation activationFn, INDArray mask)
String	tensorflowName() The opName of this function tensorflow
String	toString()

Methods inherited from class org.nd4j.autodiff.functions.DifferentialFunction

arg, args, asProperties, attributeAdaptersForFunction, calculateOutputShape, configFieldName, diff, dup, equals, f, getValue, hashCode, hasPlaceHolderInputs, isConfigProperties, larg, mappingsForFunction, onnxNames, opNum, propertiesForFunction, rarg, resolvePropertiesFromSameDiffBeforeExecution, setInstanceId, setValueFor, tensorflowNames

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

LossMultiLabel

public LossMultiLabel()

Method Detail

scoreArray

public INDArray scoreArray(INDArray labels,
                           INDArray preOutput,
                           IActivation activationFn,
                           INDArray mask)

computeScore

public double computeScore(INDArray labels,
                           INDArray preOutput,
                           IActivation activationFn,
                           INDArray mask,
                           boolean average)

Description copied from interface: ILossFunction

Compute the score (loss function value) for the given inputs.

Specified by:

computeScore in interface ILossFunction

Parameters:

labels - Label/expected preOutput

preOutput - Output of the model (neural network)

activationFn - Activation function that should be applied to preOutput

mask - Mask array; may be null

average - Whether the score should be averaged (divided by number of rows in labels/preOutput) or not @return Loss function value

computeScoreArray

public INDArray computeScoreArray(INDArray labels,
                                  INDArray preOutput,
                                  IActivation activationFn,
                                  INDArray mask)

Description copied from interface: ILossFunction

Compute the score (loss function value) for each example individually. For input [numExamples,nOut] returns scores as a column vector: [numExamples,1]

Specified by:

computeScoreArray in interface ILossFunction

Parameters:

labels - Labels/expected output

preOutput - Output of the model (neural network)

activationFn - Activation function that should be applied to preOutput

mask - @return Loss function value for each example; column vector

computeGradient

public INDArray computeGradient(INDArray labels,
                                INDArray preOutput,
                                IActivation activationFn,
                                INDArray mask)

Description copied from interface: ILossFunction

Compute the gradient of the loss function with respect to the inputs: dL/dOutput

Specified by:

computeGradient in interface ILossFunction

Parameters:

labels - Label/expected output

preOutput - Output of the model (neural network), before the activation function is applied

activationFn - Activation function that should be applied to preOutput

mask - Mask array; may be null

Returns:

Gradient dL/dPreOut

computeGradientAndScore

public org.nd4j.linalg.primitives.Pair<Double,INDArray> computeGradientAndScore(INDArray labels,
                                                                                INDArray preOutput,
                                                                                IActivation activationFn,
                                                                                INDArray mask,
                                                                                boolean average)

Description copied from interface: ILossFunction

Compute both the score (loss function value) and gradient. This is equivalent to calling ILossFunction.computeScore(INDArray, INDArray, IActivation, INDArray, boolean) and ILossFunction.computeGradient(INDArray, INDArray, IActivation, INDArray) individually

Specified by:

computeGradientAndScore in interface ILossFunction

Parameters:

labels - Label/expected output

preOutput - Output of the model (neural network)

activationFn - Activation function that should be applied to preOutput

mask - Mask array; may be null

average - Whether the score should be averaged (divided by number of rows in labels/output) or not

Returns:

The score (loss function value) and gradient

name

public String name()

Description copied from interface: ILossFunction

The opName of this function

Specified by:

name in interface ILossFunction

Returns:

toString

public String toString()

Overrides:

toString in class Object

outputVariables

public SDVariable[] outputVariables()

Description copied from class: DifferentialFunction

Return the output variables for this differential function. Note that this op *may* dynamically generate variable outputs.

Overrides:

outputVariables in class DifferentialFunction

Returns:

outputVariables

public SDVariable[] outputVariables(String baseName)

Description copied from class: DifferentialFunction

Return the output functions for this differential function.

Specified by:

outputVariables in class DifferentialFunction

Returns:

doDiff

public List<SDVariable> doDiff(List<SDVariable> f1)

Description copied from class: DifferentialFunction

The actual implementation for automatic differentiation.

Specified by:

doDiff in class DifferentialFunction

Returns:

opName

public String opName()

Description copied from class: DifferentialFunction

The name of the op

Overrides:

opName in class DifferentialFunction

Returns:

opType

public Op.Type opType()

Description copied from class: DifferentialFunction

The type of the op

Overrides:

opType in class DifferentialFunction

Returns:

initFromTensorFlow

public void initFromTensorFlow(NodeDef nodeDef,
                               SameDiff initWith,
                               Map<String,AttrValue> attributesForNode,
                               GraphDef graph)

Description copied from class: DifferentialFunction

Initialize the function from the given NodeDef

Specified by:

initFromTensorFlow in class DifferentialFunction

initFromOnnx

public void initFromOnnx(OnnxProto3.NodeProto node,
                         SameDiff initWith,
                         Map<String,OnnxProto3.AttributeProto> attributesForNode,
                         OnnxProto3.GraphProto graph)

Description copied from class: DifferentialFunction

Iniitialize the function from the given OnnxProto3.NodeProto

Specified by:

initFromOnnx in class DifferentialFunction

onnxName

public String onnxName()

Description copied from class: DifferentialFunction

The opName of this function in onnx

Specified by:

onnxName in class DifferentialFunction

Returns:

tensorflowName

public String tensorflowName()

Description copied from class: DifferentialFunction

The opName of this function tensorflow

Specified by:

tensorflowName in class DifferentialFunction

Returns: