org.deeplearning4j.datasets.iterator.impl

Class CifarDataSetIterator

java.lang.Object

org.deeplearning4j.datasets.datavec.RecordReaderDataSetIterator

org.deeplearning4j.datasets.iterator.impl.CifarDataSetIterator

All Implemented Interfaces:

Serializable, Iterator<org.nd4j.linalg.dataset.DataSet>, org.nd4j.linalg.dataset.api.iterator.DataSetIterator

public class CifarDataSetIterator
extends RecordReaderDataSetIterator

CifarDataSetIterator is an iterator for Cifar10 dataset explicitly There is a special preProcessor used to normalize the dataset based on Sergey Zagoruyko example https://github.com/szagoruyko/cifar.torch

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.deeplearning4j.datasets.datavec.RecordReaderDataSetIterator

RecordReaderDataSetIterator.Builder

Field Summary

Fields

Modifier and Type	Field and Description
protected static int	CHANNELS
protected int	exampleCount
protected static int	HEIGHT
protected org.datavec.image.transform.ImageTransform	imageTransform
protected org.datavec.image.loader.CifarLoader	loader
protected int	numExamples
protected boolean	overshot
protected boolean	train
protected boolean	useSpecialPreProcessCifar
protected static int	WIDTH

Fields inherited from class org.deeplearning4j.datasets.datavec.RecordReaderDataSetIterator

batchNum, batchSize, converter, labelIndex, labelIndexTo, last, maxNumBatches, numPossibleLabels, preProcessor, recordReader, regression, sequenceIter, useCurrent

Constructor Summary

Constructors

Constructor and Description
CifarDataSetIterator(int batchSize, int numExamples) Loads images with given batchSize & numExamples returned by the generator.
CifarDataSetIterator(int batchSize, int[] imgDim) Loads images with given batchSize & imgDim returned by the generator.
CifarDataSetIterator(int batchSize, int numExamples, boolean train) Loads images with given batchSize, numExamples, & version returned by the generator.
CifarDataSetIterator(int batchSize, int numExamples, int[] imgDim) Loads images with given batchSize, numExamples, & imgDim returned by the generator.
CifarDataSetIterator(int batchSize, int numExamples, int[] imgDim, boolean train) Loads images with given batchSize, numExamples, imgDim & version returned by the generator.
CifarDataSetIterator(int batchSize, int numExamples, int[] imgDim, boolean useSpecialPreProcessCifar, boolean train) Loads images with given batchSize, numExamples, imgDim & version returned by the generator.
CifarDataSetIterator(int batchSize, int numExamples, int[] imgDim, int numPossibleLables, org.datavec.image.transform.ImageTransform imageTransform, boolean useSpecialPreProcessCifar, boolean train) Create Cifar data specific iterator
CifarDataSetIterator(int batchSize, int numExamples, int[] imgDim, int numPossibleLables, org.datavec.image.transform.ImageTransform imageTransform, boolean useSpecialPreProcessCifar, boolean train, long rngSeed, boolean randomize) Create Cifar data specific iterator

Method Summary

Modifier and Type	Method and Description
boolean	asyncSupported()
List<String>	getLabels()
boolean	hasNext()
org.nd4j.linalg.dataset.DataSet	next(int batchSize)
void	reset()
boolean	resetSupported()
int	totalExamples()

Methods inherited from class org.deeplearning4j.datasets.datavec.RecordReaderDataSetIterator

batch, cursor, getPreProcessor, inputColumns, isCollectMetaData, loadFromMetaData, loadFromMetaData, next, numExamples, remove, setCollectMetaData, setPreProcessor, totalOutcomes

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.util.Iterator

forEachRemaining

Field Detail

HEIGHT

protected static final int HEIGHT

See Also:

Constant Field Values

WIDTH

protected static final int WIDTH

See Also:

Constant Field Values

CHANNELS

protected static final int CHANNELS

See Also:

Constant Field Values

loader

protected final org.datavec.image.loader.CifarLoader loader

numExamples

protected final int numExamples

useSpecialPreProcessCifar

protected final boolean useSpecialPreProcessCifar

train

protected final boolean train

imageTransform

protected final org.datavec.image.transform.ImageTransform imageTransform

exampleCount

protected int exampleCount

overshot

protected boolean overshot

Constructor Detail

CifarDataSetIterator

public CifarDataSetIterator(int batchSize,
                            int numExamples)

Loads images with given batchSize & numExamples returned by the generator.

CifarDataSetIterator

public CifarDataSetIterator(int batchSize,
                            int numExamples,
                            boolean train)

Loads images with given batchSize, numExamples, & version returned by the generator.

CifarDataSetIterator

public CifarDataSetIterator(int batchSize,
                            int[] imgDim)

Loads images with given batchSize & imgDim returned by the generator.

CifarDataSetIterator

public CifarDataSetIterator(int batchSize,
                            int numExamples,
                            int[] imgDim)

Loads images with given batchSize, numExamples, & imgDim returned by the generator.

CifarDataSetIterator

public CifarDataSetIterator(int batchSize,
                            int numExamples,
                            int[] imgDim,
                            boolean train)

Loads images with given batchSize, numExamples, imgDim & version returned by the generator.

CifarDataSetIterator

public CifarDataSetIterator(int batchSize,
                            int numExamples,
                            int[] imgDim,
                            boolean useSpecialPreProcessCifar,
                            boolean train)

Loads images with given batchSize, numExamples, imgDim & version returned by the generator.

CifarDataSetIterator

public CifarDataSetIterator(int batchSize,
                            int numExamples,
                            int[] imgDim,
                            int numPossibleLables,
                            org.datavec.image.transform.ImageTransform imageTransform,
                            boolean useSpecialPreProcessCifar,
                            boolean train)

Create Cifar data specific iterator

Parameters:

batchSize - the batch size of the examples

imgDim - an array of height, width and channels

numExamples - the overall number of examples

imageTransform - the transformation to apply to the images

useSpecialPreProcessCifar - use Zagoruyko's preprocess for Cifar

train - true if use training set and false for test

CifarDataSetIterator

public CifarDataSetIterator(int batchSize,
                            int numExamples,
                            int[] imgDim,
                            int numPossibleLables,
                            org.datavec.image.transform.ImageTransform imageTransform,
                            boolean useSpecialPreProcessCifar,
                            boolean train,
                            long rngSeed,
                            boolean randomize)

Create Cifar data specific iterator

Parameters:

batchSize - the batch size of the examples

imgDim - an array of height, width and channels

numExamples - the overall number of examples

imageTransform - the transformation to apply to the images

useSpecialPreProcessCifar - use Zagoruyko's preprocess for Cifar

train - true if use training set and false for test

rngSeed - Seed for RNG repeatability

randomize - If true: randomize the iteration order of the images

Method Detail

next

public org.nd4j.linalg.dataset.DataSet next(int batchSize)

Specified by:

next in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator

Overrides:

next in class RecordReaderDataSetIterator

hasNext

public boolean hasNext()

Specified by:

hasNext in interface Iterator<org.nd4j.linalg.dataset.DataSet>

Overrides:

hasNext in class RecordReaderDataSetIterator

totalExamples

public int totalExamples()

Specified by:

totalExamples in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator

Overrides:

totalExamples in class RecordReaderDataSetIterator

reset

public void reset()

Specified by:

reset in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator

Overrides:

reset in class RecordReaderDataSetIterator

resetSupported

public boolean resetSupported()

Specified by:

resetSupported in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator

Overrides:

resetSupported in class RecordReaderDataSetIterator

getLabels

public List<String> getLabels()

Specified by:

getLabels in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator

Overrides:

getLabels in class RecordReaderDataSetIterator

asyncSupported

public boolean asyncSupported()

Specified by:

asyncSupported in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator

Overrides:

asyncSupported in class RecordReaderDataSetIterator