org.deeplearning4j.datasets.iterator.file

Class BaseFileIterator<T,P>

java.lang.Object

org.deeplearning4j.datasets.iterator.file.BaseFileIterator<T,P>

Type Parameters:

T - Type of dataset

P - Type of preprocessor

All Implemented Interfaces:

Iterator<T>

Direct Known Subclasses:

FileDataSetIterator, FileMultiDataSetIterator

public abstract class BaseFileIterator<T,P>
extends Object
implements Iterator<T>

Base class for loading DataSet objects from file

Author:

Alex Black

Field Summary

Fields

Modifier and Type	Field and Description
protected int	batchSize
protected List<String>	list
protected int[]	order
protected int	position
protected P	preProcessor
protected Random	rng

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	BaseFileIterator(File[] rootDirs, boolean recursive, Random rng, int batchSize, String... validExtensions)
protected	BaseFileIterator(File rootDir, int batchSize, String... validExtensions)

Method Summary

Modifier and Type	Method and Description
protected abstract void	applyPreprocessor(T toPreProcess)
boolean	asyncSupported()
boolean	hasNext()
protected abstract T	load(File f)
protected abstract T	merge(List<T> toMerge)
protected T	mergeAndStoreRemainder(List<T> toMerge)
T	next()
void	remove()
void	reset()
boolean	resetSupported()
protected abstract int	sizeOf(T of)
protected abstract List<T>	split(T toSplit)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.util.Iterator

forEachRemaining

Field Detail

list

protected final List<String> list

batchSize

protected final int batchSize

rng

protected final Random rng

order

protected int[] order

position

protected int position

preProcessor

protected P preProcessor

Constructor Detail

BaseFileIterator

protected BaseFileIterator(@NonNull
                           File rootDir,
                           int batchSize,
                           String... validExtensions)

BaseFileIterator

protected BaseFileIterator(@NonNull
                           File[] rootDirs,
                           boolean recursive,
                           Random rng,
                           int batchSize,
                           String... validExtensions)

Method Detail

hasNext

public boolean hasNext()

Specified by:

hasNext in interface Iterator<T>

next

public T next()

Specified by:

next in interface Iterator<T>

remove

public void remove()

Specified by:

remove in interface Iterator<T>

mergeAndStoreRemainder

protected T mergeAndStoreRemainder(List<T> toMerge)

reset

public void reset()

resetSupported

public boolean resetSupported()

asyncSupported

public boolean asyncSupported()

load

protected abstract T load(File f)

sizeOf

protected abstract int sizeOf(T of)

split

protected abstract List<T> split(T toSplit)

merge

protected abstract T merge(List<T> toMerge)

applyPreprocessor

protected abstract void applyPreprocessor(T toPreProcess)