org.apache.flink.api.java.io

Class AvroInputFormat<E>

java.lang.Object

org.apache.flink.api.common.io.RichInputFormat<OT,FileInputSplit>

org.apache.flink.api.common.io.FileInputFormat<E>

org.apache.flink.api.java.io.AvroInputFormat<E>

Type Parameters:

E - the type of the result Avro record. If you specify GenericRecord then the result will be returned as a GenericRecord, so you do not have to know the schema ahead of time.

All Implemented Interfaces:

Serializable, CheckpointableInputFormat<FileInputSplit,Tuple2<Long,Long>>, InputFormat<E,FileInputSplit>, ResultTypeQueryable<E>, InputSplitSource<FileInputSplit>

public class AvroInputFormat<E>
extends FileInputFormat<E>
implements ResultTypeQueryable<E>, CheckpointableInputFormat<FileInputSplit,Tuple2<Long,Long>>

Provides a FileInputFormat for Avro records.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.api.common.io.FileInputFormat

FileInputFormat.FileBaseStatistics, FileInputFormat.InputSplitOpenThread

Field Summary

Fields inherited from class org.apache.flink.api.common.io.FileInputFormat

currentSplit, ENUMERATE_NESTED_FILES_FLAG, enumerateNestedFiles, filePath, INFLATER_INPUT_STREAM_FACTORIES, minSplitSize, numSplits, openTimeout, READ_WHOLE_SPLIT_FLAG, splitLength, splitStart, stream, unsplittable

Constructor Summary

Constructors

Constructor and Description
AvroInputFormat(Path filePath, Class<E> type)

Method Summary

Methods

Modifier and Type	Method and Description
Tuple2<Long,Long>	getCurrentState()
TypeInformation<E>	getProducedType()
long	getRecordsReadFromBlock()
E	nextRecord(E reuseValue)
void	open(FileInputSplit split)
boolean	reachedEnd()
void	reopen(FileInputSplit split, Tuple2<Long,Long> state)
void	setReuseAvroValue(boolean reuseAvroValue) Sets the flag whether to reuse the Avro value instance for all records.
void	setUnsplittable(boolean unsplittable) If set, the InputFormat will only read entire files.

Methods inherited from class org.apache.flink.api.common.io.FileInputFormat

acceptFile, close, configure, createInputSplits, decorateInputStream, extractFileExtension, getFilePath, getFileStats, getInflaterInputStreamFactory, getInputSplitAssigner, getMinSplitSize, getNestedFileEnumeration, getNumSplits, getOpenTimeout, getSplitLength, getSplitStart, getStatistics, registerInflaterInputStreamFactory, setFilePath, setFilePath, setFilesFilter, setMinSplitSize, setNestedFileEnumeration, setNumSplits, setOpenTimeout, testForUnsplittable, toString

Methods inherited from class org.apache.flink.api.common.io.RichInputFormat

closeInputFormat, getRuntimeContext, openInputFormat, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

AvroInputFormat

public AvroInputFormat(Path filePath,
               Class<E> type)

Method Detail

setReuseAvroValue

public void setReuseAvroValue(boolean reuseAvroValue)

Sets the flag whether to reuse the Avro value instance for all records. By default, the input format reuses the Avro value.

Parameters:

reuseAvroValue - True, if the input format should reuse the Avro value instance, false otherwise.

setUnsplittable

public void setUnsplittable(boolean unsplittable)

If set, the InputFormat will only read entire files.

getProducedType

public TypeInformation<E> getProducedType()

Specified by:

getProducedType in interface ResultTypeQueryable<E>

open

public void open(FileInputSplit split)
          throws IOException

Specified by:

open in interface InputFormat<E,FileInputSplit>

Overrides:

open in class FileInputFormat<E>

Throws:

IOException

reachedEnd

public boolean reachedEnd()
                   throws IOException

Specified by:

reachedEnd in interface InputFormat<E,FileInputSplit>

Throws:

IOException

getRecordsReadFromBlock

public long getRecordsReadFromBlock()

nextRecord

public E nextRecord(E reuseValue)
             throws IOException

Specified by:

nextRecord in interface InputFormat<E,FileInputSplit>

Throws:

IOException

getCurrentState

public Tuple2<Long,Long> getCurrentState()
                                  throws IOException

Specified by:

getCurrentState in interface CheckpointableInputFormat<FileInputSplit,Tuple2<Long,Long>>

Throws:

IOException

reopen

public void reopen(FileInputSplit split,
          Tuple2<Long,Long> state)
            throws IOException

Specified by:

reopen in interface CheckpointableInputFormat<FileInputSplit,Tuple2<Long,Long>>

Throws:

IOException