org.apache.flink.runtime.io.network.api.writer

Class BroadcastRecordWriter<T extends org.apache.flink.core.io.IOReadableWritable>

java.lang.Object

org.apache.flink.runtime.io.network.api.writer.RecordWriter<T>

org.apache.flink.runtime.io.network.api.writer.BroadcastRecordWriter<T>

Type Parameters:

T - the type of the record that can be emitted with this record writer

All Implemented Interfaces:

AvailabilityProvider

public final class BroadcastRecordWriter<T extends org.apache.flink.core.io.IOReadableWritable>
extends RecordWriter<T>

A special record-oriented runtime result writer only for broadcast mode.

The BroadcastRecordWriter extends the RecordWriter and maintain a single BufferBuilder for all the channels. Then the serialization results need be copied only once to this buffer which would be shared for all the channels in a more efficient way.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.runtime.io.AvailabilityProvider

AvailabilityProvider.AvailabilityHelper

Field Summary

Fields inherited from class org.apache.flink.runtime.io.network.api.writer.RecordWriter

DEFAULT_OUTPUT_FLUSH_THREAD_NAME, numberOfChannels, rng, serializer, targetPartition

Fields inherited from interface org.apache.flink.runtime.io.AvailabilityProvider

AVAILABLE

Method Summary

Modifier and Type	Method and Description
void	broadcastEmit(T record) This is used to broadcast streaming Watermarks in-band with records.
void	clearBuffers() Closes the BufferBuilders for all the channels.
void	closeBufferBuilder(int targetChannel) Marks the current BufferBuilder as finished and releases the resources for the target channel.
void	emit(T record) This is used to send regular records.
void	emptyCurrentBufferBuilder(int targetChannel) Marks the current BufferBuilder as empty for the target channel.
void	flushTargetPartition(int targetChannel) The flush could be triggered by randomEmit(IOReadableWritable), emit(IOReadableWritable) or broadcastEmit(IOReadableWritable).
BufferBuilder	getBufferBuilder(int targetChannel) The BufferBuilder may already exist if not filled up last time, otherwise we need request a new one for this target channel.
void	randomEmit(T record) This is used to send LatencyMarks to a random target channel.
BufferBuilder	requestNewBufferBuilder(int targetChannel) The request could be from broadcast or non-broadcast modes like randomEmit(IOReadableWritable).
void	tryFinishCurrentBufferBuilder(int targetChannel) Marks the current BufferBuilder as finished if present and clears the state for next one.

Methods inherited from class org.apache.flink.runtime.io.network.api.writer.RecordWriter

broadcastEvent, checkErroneous, close, copyFromSerializerToTargetChannel, emit, finishBufferBuilder, flushAll, getAvailableFuture, setMetricGroup

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.runtime.io.AvailabilityProvider

isApproximatelyAvailable, isAvailable

Method Detail

emit

public void emit(T record)
          throws IOException,
                 InterruptedException

Description copied from class: RecordWriter

This is used to send regular records.

Specified by:

emit in class RecordWriter<T extends org.apache.flink.core.io.IOReadableWritable>

Throws:

IOException

InterruptedException

randomEmit

public void randomEmit(T record)
                throws IOException,
                       InterruptedException

Description copied from class: RecordWriter

This is used to send LatencyMarks to a random target channel.

Specified by:

randomEmit in class RecordWriter<T extends org.apache.flink.core.io.IOReadableWritable>

Throws:

IOException

InterruptedException

broadcastEmit

public void broadcastEmit(T record)
                   throws IOException,
                          InterruptedException

Description copied from class: RecordWriter

This is used to broadcast streaming Watermarks in-band with records.

Specified by:

broadcastEmit in class RecordWriter<T extends org.apache.flink.core.io.IOReadableWritable>

Throws:

IOException

InterruptedException

flushTargetPartition

public void flushTargetPartition(int targetChannel)

The flush could be triggered by randomEmit(IOReadableWritable), emit(IOReadableWritable) or broadcastEmit(IOReadableWritable). Only random emit should flush a single target channel, otherwise we should flush all the channels.

Overrides:

flushTargetPartition in class RecordWriter<T extends org.apache.flink.core.io.IOReadableWritable>

getBufferBuilder

public BufferBuilder getBufferBuilder(int targetChannel)
                               throws IOException,
                                      InterruptedException

Description copied from class: RecordWriter

The BufferBuilder may already exist if not filled up last time, otherwise we need request a new one for this target channel.

Throws:

IOException

InterruptedException

requestNewBufferBuilder

public BufferBuilder requestNewBufferBuilder(int targetChannel)
                                      throws IOException,
                                             InterruptedException

The request could be from broadcast or non-broadcast modes like randomEmit(IOReadableWritable).

For non-broadcast, the created BufferConsumer is only for the target channel.

For broadcast, all the channels share the same requested BufferBuilder and the created BufferConsumer is copied for every channel.

Throws:

IOException

InterruptedException

tryFinishCurrentBufferBuilder

public void tryFinishCurrentBufferBuilder(int targetChannel)

Description copied from class: RecordWriter

Marks the current BufferBuilder as finished if present and clears the state for next one.

emptyCurrentBufferBuilder

public void emptyCurrentBufferBuilder(int targetChannel)

Description copied from class: RecordWriter

Marks the current BufferBuilder as empty for the target channel.

closeBufferBuilder

public void closeBufferBuilder(int targetChannel)

Description copied from class: RecordWriter

Marks the current BufferBuilder as finished and releases the resources for the target channel.

clearBuffers

public void clearBuffers()

Description copied from class: RecordWriter

Closes the BufferBuilders for all the channels.

Specified by:

clearBuffers in class RecordWriter<T extends org.apache.flink.core.io.IOReadableWritable>