public class PipelinedResultPartition extends BufferWritingResultPartition implements CheckpointedResultPartition, ChannelStateHolder
This result partition implementation is used both in batch and streaming. For streaming it supports low latency transfers (ensure data is sent within x milliseconds) or unconstrained while for batch it transfers only once a buffer is full. Additionally, for streaming use this typically limits the length of the buffer backlog to not have too much data in flight, while for batch we do not constrain this.
The PipelinedResultPartition cannot reconnect once a consumer disconnects (finished or
errored). Once all consumers have disconnected (released the subpartition, notified via the call
onConsumedSubpartition(int)) then the partition as a whole is disposed and all buffers
are freed.
AvailabilityProvider.AvailabilityHelpersubpartitionsbufferCompressor, bufferPool, LOG, numBuffersOut, numBytesOut, numSubpartitions, partitionId, partitionManager, partitionType, resultPartitionBytesAVAILABLE| 构造器和说明 |
|---|
PipelinedResultPartition(String owningTaskName,
int partitionIndex,
ResultPartitionID partitionId,
ResultPartitionType partitionType,
ResultSubpartition[] subpartitions,
int numTargetKeyGroups,
ResultPartitionManager partitionManager,
BufferCompressor bufferCompressor,
org.apache.flink.util.function.SupplierWithException<BufferPool,IOException> bufferPoolFactory) |
| 限定符和类型 | 方法和说明 |
|---|---|
void |
close()
Closes the partition writer which releases the allocated resource, for example the buffer
pool.
|
void |
finishReadRecoveredState(boolean notifyAndBlockOnCompletion) |
void |
flush(int targetSubpartition)
Manually trigger the consumption of data from the given subpartitions.
|
void |
flushAll()
Manually trigger the consumption of data from all subpartitions.
|
CompletableFuture<Void> |
getAllDataProcessedFuture()
Gets the future indicating whether all the records has been processed by the downstream
tasks.
|
CheckpointedResultSubpartition |
getCheckpointedSubpartition(int subpartitionIndex)
Gets the checkpointed subpartition with the given subpartitionIndex.
|
void |
notifyEndOfData(StopMode mode)
Notifies the downstream tasks that this
ResultPartitionWriter have emitted all the
user records. |
void |
onSubpartitionAllDataProcessed(int subpartition)
The subpartition notifies that the corresponding downstream task have processed all the user
records.
|
void |
setChannelStateWriter(ChannelStateWriter channelStateWriter)
Injects the
ChannelStateWriter. |
String |
toString() |
abortCheckpoint, alignedBarrierTimeout, broadcastEvent, broadcastRecord, createSubpartitionView, emitRecord, finish, flushAllSubpartitions, flushSubpartition, getAllPartitions, getHardBackPressuredTimeMsPerSecond, getNumberOfQueuedBuffers, getNumberOfQueuedBuffers, getSizeOfQueuedBuffersUnsafe, releaseInternal, setMetricGroup, setupInternalcanBeCompressed, checkInProduceState, fail, getAvailableFuture, getBufferPool, getFailureCause, getNumberOfSubpartitions, getNumTargetKeyGroups, getOwningTaskName, getPartitionId, getPartitionIndex, getPartitionManager, getPartitionType, isFinished, isReleased, release, release, setMaxOverdraftBuffersPerGate, setupclone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, waitand, isApproximatelyAvailable, isAvailable, orpublic PipelinedResultPartition(String owningTaskName, int partitionIndex, ResultPartitionID partitionId, ResultPartitionType partitionType, ResultSubpartition[] subpartitions, int numTargetKeyGroups, ResultPartitionManager partitionManager, @Nullable BufferCompressor bufferCompressor, org.apache.flink.util.function.SupplierWithException<BufferPool,IOException> bufferPoolFactory)
public void setChannelStateWriter(ChannelStateWriter channelStateWriter)
ChannelStateHolderChannelStateWriter. Must only be called once.setChannelStateWriter 在接口中 ChannelStateHolderpublic CheckpointedResultSubpartition getCheckpointedSubpartition(int subpartitionIndex)
CheckpointedResultPartitionpublic void flushAll()
ResultPartitionWriterflushAll 在接口中 ResultPartitionWriterpublic void flush(int targetSubpartition)
ResultPartitionWriterflush 在接口中 ResultPartitionWriterpublic void notifyEndOfData(StopMode mode) throws IOException
ResultPartitionWriterResultPartitionWriter have emitted all the
user records.notifyEndOfData 在接口中 ResultPartitionWriternotifyEndOfData 在类中 ResultPartitionmode - tells if we should flush all records or not (it is false in case of
stop-with-savepoint (--no-drain))IOExceptionpublic CompletableFuture<Void> getAllDataProcessedFuture()
ResultPartitionWriterpublic void onSubpartitionAllDataProcessed(int subpartition)
ResultPartitiononSubpartitionAllDataProcessed 在类中 ResultPartitionsubpartition - The index of the subpartition sending the notification.EndOfDatapublic String toString()
toString 在类中 ResultPartitionpublic void finishReadRecoveredState(boolean notifyAndBlockOnCompletion)
throws IOException
finishReadRecoveredState 在接口中 CheckpointedResultPartitionIOExceptionpublic void close()
ResultPartitionWriterclose 在接口中 AutoCloseableclose 在接口中 ResultPartitionWriterclose 在类中 BufferWritingResultPartitionCopyright © 2014–2023 The Apache Software Foundation. All rights reserved.