org.apache.flink.runtime.io.network.api.writer

Class RecordWriter<T extends org.apache.flink.core.io.IOReadableWritable>

java.lang.Object

org.apache.flink.runtime.io.network.api.writer.RecordWriter<T>

Type Parameters:

T - the type of the record that can be emitted with this record writer

All Implemented Interfaces:

AvailabilityProvider

Direct Known Subclasses:

BroadcastRecordWriter, ChannelSelectorRecordWriter

public abstract class RecordWriter<T extends org.apache.flink.core.io.IOReadableWritable>
extends Object
implements AvailabilityProvider

An abstract record-oriented runtime result writer.

The RecordWriter wraps the runtime's ResultPartitionWriter and takes care of serializing records into buffers.

Important: it is necessary to call flushAll() after all records have been written with emit(IOReadableWritable). This ensures that all produced records are written to the output stream (incl. partially filled ones).

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.runtime.io.AvailabilityProvider

AvailabilityProvider.AvailabilityHelper

Field Summary

Fields

Modifier and Type	Field and Description
static String	DEFAULT_OUTPUT_FLUSH_THREAD_NAME Default name for the output flush thread, if no name with a task reference is given.
protected int	numberOfChannels
protected Random	rng
protected RecordSerializer<T>	serializer
protected ResultPartitionWriter	targetPartition

Fields inherited from interface org.apache.flink.runtime.io.AvailabilityProvider

AVAILABLE

Method Summary

Modifier and Type	Method and Description
abstract void	broadcastEmit(T record) This is used to broadcast streaming Watermarks in-band with records.
void	broadcastEvent(AbstractEvent event)
protected void	checkErroneous()
abstract void	clearBuffers() Closes the BufferBuilders for all the channels.
void	close() Closes the writer.
protected boolean	copyFromSerializerToTargetChannel(int targetChannel)
abstract void	emit(T record) This is used to send regular records.
protected void	emit(T record, int targetChannel)
protected void	finishBufferBuilder(BufferBuilder bufferBuilder)
void	flushAll()
protected void	flushTargetPartition(int targetChannel)
CompletableFuture<?>	getAvailableFuture()
abstract void	randomEmit(T record) This is used to send LatencyMarks to a random target channel.
void	setMetricGroup(TaskIOMetricGroup metrics) Sets the metric group for this RecordWriter.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.runtime.io.AvailabilityProvider

isApproximatelyAvailable, isAvailable

Field Detail

DEFAULT_OUTPUT_FLUSH_THREAD_NAME

@VisibleForTesting
public static final String DEFAULT_OUTPUT_FLUSH_THREAD_NAME

Default name for the output flush thread, if no name with a task reference is given.

See Also:

Constant Field Values

targetPartition

protected final ResultPartitionWriter targetPartition

numberOfChannels

protected final int numberOfChannels

serializer

protected final RecordSerializer<T extends org.apache.flink.core.io.IOReadableWritable> serializer

rng

protected final Random rng

Method Detail

emit

protected void emit(T record,
                    int targetChannel)
             throws IOException,
                    InterruptedException

Throws:

IOException

InterruptedException

copyFromSerializerToTargetChannel

protected boolean copyFromSerializerToTargetChannel(int targetChannel)
                                             throws IOException,
                                                    InterruptedException

Parameters:

targetChannel -

Returns:

true if the intermediate serialization buffer should be pruned

Throws:

IOException

InterruptedException

broadcastEvent

public void broadcastEvent(AbstractEvent event)
                    throws IOException

Throws:

IOException

flushAll

public void flushAll()

flushTargetPartition

protected void flushTargetPartition(int targetChannel)

setMetricGroup

public void setMetricGroup(TaskIOMetricGroup metrics)

Sets the metric group for this RecordWriter.

finishBufferBuilder

protected void finishBufferBuilder(BufferBuilder bufferBuilder)

getAvailableFuture

public CompletableFuture<?> getAvailableFuture()

Specified by:

getAvailableFuture in interface AvailabilityProvider

Returns:

a future that is completed if the respective provider is available.

emit

public abstract void emit(T record)
                   throws IOException,
                          InterruptedException

This is used to send regular records.

Throws:

IOException

InterruptedException

randomEmit

public abstract void randomEmit(T record)
                         throws IOException,
                                InterruptedException

This is used to send LatencyMarks to a random target channel.

Throws:

IOException

InterruptedException

broadcastEmit

public abstract void broadcastEmit(T record)
                            throws IOException,
                                   InterruptedException

This is used to broadcast streaming Watermarks in-band with records.

Throws:

IOException

InterruptedException

clearBuffers

public abstract void clearBuffers()

Closes the BufferBuilders for all the channels.

close

public void close()

Closes the writer. This stops the flushing thread (if there is one).

checkErroneous

protected void checkErroneous()
                       throws IOException

Throws:

IOException