org.apache.flink.runtime.io.network.partition

类 PipelinedSubpartition

java.lang.Object

org.apache.flink.runtime.io.network.partition.ResultSubpartition

org.apache.flink.runtime.io.network.partition.PipelinedSubpartition

所有已实现的接口:

ChannelStateHolder, CheckpointedResultSubpartition

直接已知子类:

PipelinedApproximateSubpartition

public class PipelinedSubpartition
extends ResultSubpartition
implements CheckpointedResultSubpartition, ChannelStateHolder

A pipelined in-memory only subpartition, which can be consumed once.

Whenever ResultSubpartition.add(BufferConsumer) adds a finished BufferConsumer or a second BufferConsumer (in which case we will assume the first one finished), we will notify a read view created via ResultSubpartition.createReadView(BufferAvailabilityListener) of new data availability. Except by calling flush() explicitly, we always only notify when the first finished buffer turns up and then, the reader has to drain the buffers via pollBuffer() until its return value shows no more buffers being available. This results in a buffer queue which is either empty or has an unfinished BufferConsumer left from which the notifications will eventually start again.

Explicit calls to flush() will force this notification for any BufferConsumer present in the queue.

嵌套类概要

从类继承的嵌套类/接口 org.apache.flink.runtime.io.network.partition.ResultSubpartition

ResultSubpartition.BufferAndBacklog

字段概要

从类继承的字段 org.apache.flink.runtime.io.network.partition.ResultSubpartition

parent, subpartitionInfo

方法概要

限定符和类型	方法和说明
void	abortCheckpoint(long checkpointId, CheckpointException cause)
void	acknowledgeAllDataProcessed()
int	add(BufferConsumer bufferConsumer, int partialRecordLength) Adds the given buffer.
void	addRecovered(BufferConsumer bufferConsumer)
void	alignedBarrierTimeout(long checkpointId)
void	bufferSize(int desirableNewBufferSize)
PipelinedSubpartitionView	createReadView(BufferAvailabilityListener availabilityListener)
void	finish()
void	finishReadRecoveredState(boolean notifyAndBlockOnCompletion)
void	flush()
ResultSubpartitionView.AvailabilityWithBacklog	getAvailabilityAndBacklog(int numCreditsAvailable)
int	getBuffersInBacklogUnsafe() Gets the number of non-event buffers in this subpartition.
long	getChannelStateCheckpointId()
int	getNumberOfQueuedBuffers() Get the current size of the queue.
protected long	getTotalNumberOfBuffersUnsafe() Gets the total numbers of buffers (data buffers plus events).
protected long	getTotalNumberOfBytesUnsafe()
boolean	isReleased()
void	release()
BufferBuilder	requestBufferBuilderBlocking()
void	setChannelStateWriter(ChannelStateWriter channelStateWriter) Injects the ChannelStateWriter.
String	toString()
int	unsynchronizedGetNumberOfQueuedBuffers() Makes a best effort to get the current size of the queue.

从类继承的方法 org.apache.flink.runtime.io.network.partition.ResultSubpartition

add, getSubPartitionIndex, getSubpartitionInfo, onConsumedSubpartition

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

从接口继承的方法 org.apache.flink.runtime.io.network.partition.CheckpointedResultSubpartition

getSubpartitionInfo

方法详细资料

setChannelStateWriter

public void setChannelStateWriter(ChannelStateWriter channelStateWriter)

从接口复制的说明: ChannelStateHolder

Injects the ChannelStateWriter. Must only be called once.

指定者:

setChannelStateWriter 在接口中 ChannelStateHolder

add

public int add(BufferConsumer bufferConsumer,
               int partialRecordLength)

从类复制的说明: ResultSubpartition

Adds the given buffer.

The request may be executed synchronously, or asynchronously, depending on the implementation.

IMPORTANT: Before adding new BufferConsumer previously added must be in finished state. Because of the performance reasons, this is only enforced during the data reading. Priority events can be added while the previous buffer consumer is still open, in which case the open buffer consumer is overtaken.

指定者:

add 在类中 ResultSubpartition

参数:

bufferConsumer - the buffer to add (transferring ownership to this writer)

partialRecordLength - the length of bytes to skip in order to start with a complete record, from position index 0 of the underlying .

返回:

the preferable buffer size for this subpartition or -1 if the add operation fails.

addRecovered

public void addRecovered(BufferConsumer bufferConsumer)
                  throws IOException

指定者:

addRecovered 在接口中 CheckpointedResultSubpartition

抛出:

IOException

finishReadRecoveredState

public void finishReadRecoveredState(boolean notifyAndBlockOnCompletion)
                              throws IOException

指定者:

finishReadRecoveredState 在接口中 CheckpointedResultSubpartition

抛出:

IOException

finish

public void finish()
            throws IOException

指定者:

finish 在类中 ResultSubpartition

抛出:

IOException

alignedBarrierTimeout

public void alignedBarrierTimeout(long checkpointId)
                           throws IOException

指定者:

alignedBarrierTimeout 在类中 ResultSubpartition

抛出:

IOException

abortCheckpoint

public void abortCheckpoint(long checkpointId,
                            CheckpointException cause)

指定者:

abortCheckpoint 在类中 ResultSubpartition

release

public void release()

指定者:

release 在类中 ResultSubpartition

acknowledgeAllDataProcessed

public void acknowledgeAllDataProcessed()

isReleased

public boolean isReleased()

指定者:

isReleased 在类中 ResultSubpartition

createReadView

public PipelinedSubpartitionView createReadView(BufferAvailabilityListener availabilityListener)

指定者:

createReadView 在类中 ResultSubpartition

getAvailabilityAndBacklog

public ResultSubpartitionView.AvailabilityWithBacklog getAvailabilityAndBacklog(int numCreditsAvailable)

getNumberOfQueuedBuffers

public int getNumberOfQueuedBuffers()

从类复制的说明: ResultSubpartition

Get the current size of the queue.

指定者:

getNumberOfQueuedBuffers 在类中 ResultSubpartition

bufferSize

public void bufferSize(int desirableNewBufferSize)

指定者:

bufferSize 在类中 ResultSubpartition

toString

public String toString()

覆盖:

toString 在类中 Object

unsynchronizedGetNumberOfQueuedBuffers

public int unsynchronizedGetNumberOfQueuedBuffers()

从类复制的说明: ResultSubpartition

Makes a best effort to get the current size of the queue. This method must not acquire locks or interfere with the task and network threads in any way.

指定者:

unsynchronizedGetNumberOfQueuedBuffers 在类中 ResultSubpartition

flush

public void flush()

指定者:

flush 在类中 ResultSubpartition

getTotalNumberOfBuffersUnsafe

protected long getTotalNumberOfBuffersUnsafe()

从类复制的说明: ResultSubpartition

Gets the total numbers of buffers (data buffers plus events).

指定者:

getTotalNumberOfBuffersUnsafe 在类中 ResultSubpartition

getTotalNumberOfBytesUnsafe

protected long getTotalNumberOfBytesUnsafe()

指定者:

getTotalNumberOfBytesUnsafe 在类中 ResultSubpartition

getBuffersInBacklogUnsafe

public int getBuffersInBacklogUnsafe()

Gets the number of non-event buffers in this subpartition.

requestBufferBuilderBlocking

public BufferBuilder requestBufferBuilderBlocking()
                                           throws InterruptedException

指定者:

requestBufferBuilderBlocking 在接口中 CheckpointedResultSubpartition

抛出:

InterruptedException

getChannelStateCheckpointId

@VisibleForTesting
public long getChannelStateCheckpointId()