ai.djl.nn.recurrent

Class LSTM

java.lang.Object

ai.djl.nn.AbstractBlock

ai.djl.nn.recurrent.RecurrentBlock

ai.djl.nn.recurrent.LSTM

All Implemented Interfaces:

Block

public class LSTM
extends RecurrentBlock

LSTM is an implementation of recurrent neural networks which applies Long Short-Term Memory recurrent layer to input.

Reference paper - LONG SHORT-TERM MEMORY - Hochreiter, 1997. http://www.bioinf.jku.at/publications/older/2604.pdf

The LSTM operator is formulated as below:

$$ \begin{split}\begin{array}{ll} i_t = \mathrm{sigmoid}(W_{ii} x_t + b_{ii} + W_{hi} h_{(t-1)} + b_{hi}) \\ f_t = \mathrm{sigmoid}(W_{if} x_t + b_{if} + W_{hf} h_{(t-1)} + b_{hf}) \\ g_t = \tanh(W_{ig} x_t + b_{ig} + W_{hc} h_{(t-1)} + b_{hg}) \\ o_t = \mathrm{sigmoid}(W_{io} x_t + b_{io} + W_{ho} h_{(t-1)} + b_{ho}) \\ c_t = f_t * c_{(t-1)} + i_t * g_t \\ h_t = o_t * \tanh(c_t) \end{array}\end{split} $$

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	LSTM.Builder The Builder to construct a LSTM type of Block.

Nested classes/interfaces inherited from class ai.djl.nn.recurrent.RecurrentBlock

RecurrentBlock.BaseBuilder<T extends RecurrentBlock.BaseBuilder>

Field Summary

Fields inherited from class ai.djl.nn.recurrent.RecurrentBlock

beginState, dropRate, gates, mode, numDirections, numStackedLayers, stateOutputs, stateSize, useSequenceLength

Fields inherited from class ai.djl.nn.AbstractBlock

children, inputNames, inputShapes, parameters, parameterShapeCallbacks, version

Method Summary

Modifier and Type	Method and Description
static LSTM.Builder	builder() Creates a builder to build a LSTM.
NDList	forward(ParameterStore parameterStore, NDList inputs, boolean training, ai.djl.util.PairList<java.lang.String,java.lang.Object> params) Applies the operating function of the block once.
protected NDList	opInputs(ParameterStore parameterStore, NDList inputs, boolean training)
protected void	resetBeginStates()
void	setBeginStates(NDList beginStates) Sets the initial NDArray value for the hidden states.

Methods inherited from class ai.djl.nn.recurrent.RecurrentBlock

beforeInitialize, getOutputShapes, getParameterShape, isBidirectional, loadMetadata, setStateOutputs, updateInputLayoutToTNC, validateInputSize

Methods inherited from class ai.djl.nn.AbstractBlock

addChildBlock, addParameter, addParameter, addParameter, cast, clear, describeInput, getChildren, getDirectParameters, getParameters, initialize, initializeChildBlocks, isInitialized, loadParameters, readInputShapes, saveInputShapes, saveMetadata, saveParameters, setInitializer, setInitializer, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface ai.djl.nn.Block

forward, forward, validateLayout

Method Detail

forward

public NDList forward(ParameterStore parameterStore,
                      NDList inputs,
                      boolean training,
                      ai.djl.util.PairList<java.lang.String,java.lang.Object> params)

Applies the operating function of the block once. This method should be called only on blocks that are initialized.

Specified by:

forward in interface Block

Overrides:

forward in class RecurrentBlock

Parameters:

parameterStore - the parameter store

inputs - the input NDList

training - true for a training forward pass

params - optional parameters

Returns:

the output of the forward pass

setBeginStates

public void setBeginStates(NDList beginStates)

Sets the initial NDArray value for the hidden states.

Overrides:

setBeginStates in class RecurrentBlock

Parameters:

beginStates - the NDArray value for the hidden states

resetBeginStates

protected void resetBeginStates()

Overrides:

resetBeginStates in class RecurrentBlock

opInputs

protected NDList opInputs(ParameterStore parameterStore,
                          NDList inputs,
                          boolean training)

Overrides:

opInputs in class RecurrentBlock

builder

public static LSTM.Builder builder()

Creates a builder to build a LSTM.

Returns:

a new builder