org.deeplearning4j.datasets.iterator

Class DataSetIteratorSplitter

java.lang.Object

org.deeplearning4j.datasets.iterator.DataSetIteratorSplitter

public class DataSetIteratorSplitter
extends Object

This iterator virtually splits given MultiDataSetIterator into Train and Test parts. I.e. you have 100000 examples. Your batch size is 32. That means you have 3125 total batches. With split ratio of 0.7 that will give you 2187 training batches, and 938 test batches. PLEASE NOTE: You can't use Test iterator twice in a row. Train iterator should be used before Test iterator use. PLEASE NOTE: You can't use this iterator, if underlying iterator uses randomization/shuffle between epochs.

Author:

raver119@gmail.com

Field Summary

Fields

Modifier and Type	Field and Description
protected org.nd4j.linalg.dataset.api.iterator.DataSetIterator	backedIterator
protected AtomicLong	counter
protected org.nd4j.linalg.dataset.DataSet	firstTrain
protected long	numTest
protected long	numTrain
protected double	ratio
protected AtomicBoolean	resetPending
protected long	totalExamples

Constructor Summary

Constructors

Constructor and Description
DataSetIteratorSplitter(org.nd4j.linalg.dataset.api.iterator.DataSetIterator baseIterator, long totalExamples, double ratio)

Method Summary

Modifier and Type	Method and Description
org.nd4j.linalg.dataset.api.iterator.DataSetIterator	getTestIterator() This method returns test iterator instance
org.nd4j.linalg.dataset.api.iterator.DataSetIterator	getTrainIterator() This method returns train iterator instance

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

backedIterator

protected org.nd4j.linalg.dataset.api.iterator.DataSetIterator backedIterator

totalExamples

protected final long totalExamples

ratio

protected final double ratio

numTrain

protected final long numTrain

numTest

protected final long numTest

counter

protected AtomicLong counter

resetPending

protected AtomicBoolean resetPending

firstTrain

protected org.nd4j.linalg.dataset.DataSet firstTrain

Constructor Detail

DataSetIteratorSplitter

public DataSetIteratorSplitter(@NonNull
                               org.nd4j.linalg.dataset.api.iterator.DataSetIterator baseIterator,
                               long totalExamples,
                               double ratio)

Method Detail

getTrainIterator

public org.nd4j.linalg.dataset.api.iterator.DataSetIterator getTrainIterator()

This method returns train iterator instance

Returns:

getTestIterator

public org.nd4j.linalg.dataset.api.iterator.DataSetIterator getTestIterator()

This method returns test iterator instance

Returns: