org.deeplearning4j.datasets.iterator

Class MultiDataSetIteratorSplitter

java.lang.Object

org.deeplearning4j.datasets.iterator.MultiDataSetIteratorSplitter

public class MultiDataSetIteratorSplitter
extends Object

This iterator virtually splits given MultiDataSetIterator into Train and Test parts. I.e. you have 100000 examples. Your batch size is 32. That means you have 3125 total batches. With split ratio of 0.7 that will give you 2187 training batches, and 938 test batches. PLEASE NOTE: You can't use Test iterator twice in a row. Train iterator should be used before Test iterator use. PLEASE NOTE: You can't use this iterator, if underlying iterator uses randomization/shuffle between epochs.

Author:

raver119@gmail.com

Field Summary

Fields

Modifier and Type	Field and Description
protected org.nd4j.linalg.dataset.api.iterator.MultiDataSetIterator	backedIterator
protected AtomicLong	counter
protected org.nd4j.linalg.dataset.MultiDataSet	firstTrain
protected long	numTest
protected long	numTrain
protected double	ratio
protected AtomicBoolean	resetPending
protected long	totalExamples

Constructor Summary

Constructors

Constructor and Description
MultiDataSetIteratorSplitter(org.nd4j.linalg.dataset.api.iterator.MultiDataSetIterator baseIterator, long totalBatches, double ratio)

Method Summary

Modifier and Type	Method and Description
org.nd4j.linalg.dataset.api.iterator.MultiDataSetIterator	getTestIterator() This method returns test iterator instance
org.nd4j.linalg.dataset.api.iterator.MultiDataSetIterator	getTrainIterator() This method returns train iterator instance

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

backedIterator

protected org.nd4j.linalg.dataset.api.iterator.MultiDataSetIterator backedIterator

totalExamples

protected final long totalExamples

ratio

protected final double ratio

numTrain

protected final long numTrain

numTest

protected final long numTest

counter

protected AtomicLong counter

resetPending

protected AtomicBoolean resetPending

firstTrain

protected org.nd4j.linalg.dataset.MultiDataSet firstTrain

Constructor Detail

MultiDataSetIteratorSplitter

public MultiDataSetIteratorSplitter(@NonNull
                                    org.nd4j.linalg.dataset.api.iterator.MultiDataSetIterator baseIterator,
                                    long totalBatches,
                                    double ratio)

Parameters:

baseIterator -

totalBatches - - total number of batches in underlying iterator. this value will be used to determine number of test/train batches

ratio - - this value will be used as splitter. should be between in range of 0.0 > X < 1.0. I.e. if value 0.7 is provided, then 70% of total examples will be used for training, and 30% of total examples will be used for testing

Method Detail

getTrainIterator

public org.nd4j.linalg.dataset.api.iterator.MultiDataSetIterator getTrainIterator()

This method returns train iterator instance

Returns:

getTestIterator

public org.nd4j.linalg.dataset.api.iterator.MultiDataSetIterator getTestIterator()

This method returns test iterator instance

Returns: