org.apache.flink.runtime.io.network.partition.hybrid

Class HsSubpartitionMemoryDataManager

java.lang.Object

org.apache.flink.runtime.io.network.partition.hybrid.HsSubpartitionMemoryDataManager

All Implemented Interfaces:

HsDataView

public class HsSubpartitionMemoryDataManager
extends Object
implements HsDataView

This class is responsible for managing the data in a single subpartition. One HsMemoryDataManager will hold multiple HsSubpartitionMemoryDataManager.

Method Summary

Modifier and Type	Method and Description
void	append(ByteBuffer record, Buffer.DataType dataType) Append record to HsSubpartitionMemoryDataManager.
Optional<ResultSubpartition.BufferAndBacklog>	consumeBuffer(int toConsumeIndex) Check whether the head of unConsumedBuffers is the buffer to be consumed.
int	getBacklog() Get the number of buffers backlog.
Deque<BufferIndexAndChannel>	getBuffersSatisfyStatus(HsSpillingInfoProvider.SpillStatus spillStatus, HsSpillingInfoProvider.ConsumeStatus consumeStatus) Get buffers in allBuffers that satisfy expected HsSpillingInfoProvider.SpillStatus and HsSpillingInfoProvider.ConsumeStatus.
Buffer.DataType	peekNextToConsumeDataType(int nextToConsumeIndex) Check whether the head of unConsumedBuffers is the buffer to be consumed next time.
void	releaseDataView() Release this HsDataView when related subpartition view is releasing.
void	releaseSubpartitionBuffers(List<BufferIndexAndChannel> toRelease) Release this subpartition's buffers in a decision.
void	setOutputMetrics(HsOutputMetrics outputMetrics)
List<BufferWithIdentity>	spillSubpartitionBuffers(List<BufferIndexAndChannel> toSpill, CompletableFuture<Void> spillDoneFuture) Spill this subpartition's buffers in a decision.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

peekNextToConsumeDataType

public Buffer.DataType peekNextToConsumeDataType(int nextToConsumeIndex)

Check whether the head of unConsumedBuffers is the buffer to be consumed next time. If so, return the next buffer's data type.

Specified by:

peekNextToConsumeDataType in interface HsDataView

Parameters:

nextToConsumeIndex - index of the buffer to be consumed next time.

Returns:

If the head of unConsumedBuffers is target, return the buffer's data type. Otherwise, return Buffer.DataType.NONE.

consumeBuffer

public Optional<ResultSubpartition.BufferAndBacklog> consumeBuffer(int toConsumeIndex)

Check whether the head of unConsumedBuffers is the buffer to be consumed. If so, return the buffer and backlog.

Specified by:

consumeBuffer in interface HsDataView

Parameters:

toConsumeIndex - index of buffer to be consumed.

Returns:

If the head of unConsumedBuffers is target, return optional of the buffer and backlog. Otherwise, return Optional.empty().

getBacklog

public int getBacklog()

Description copied from interface: HsDataView

Get the number of buffers backlog.

Specified by:

getBacklog in interface HsDataView

Returns:

backlog of this view's corresponding subpartition.

releaseDataView

public void releaseDataView()

Description copied from interface: HsDataView

Release this HsDataView when related subpartition view is releasing.

Specified by:

releaseDataView in interface HsDataView

append

public void append(ByteBuffer record,
                   Buffer.DataType dataType)
            throws InterruptedException

Append record to HsSubpartitionMemoryDataManager.

Parameters:

record - to be managed by this class.

dataType - the type of this record. In other words, is it data or event.

Throws:

InterruptedException

getBuffersSatisfyStatus

public Deque<BufferIndexAndChannel> getBuffersSatisfyStatus(HsSpillingInfoProvider.SpillStatus spillStatus,
                                                            HsSpillingInfoProvider.ConsumeStatus consumeStatus)

Get buffers in allBuffers that satisfy expected HsSpillingInfoProvider.SpillStatus and HsSpillingInfoProvider.ConsumeStatus.

Parameters:

spillStatus - the status of spilling expected.

consumeStatus - the status of consuming expected.

Returns:

buffers satisfy expected status in order.

spillSubpartitionBuffers

public List<BufferWithIdentity> spillSubpartitionBuffers(List<BufferIndexAndChannel> toSpill,
                                                         CompletableFuture<Void> spillDoneFuture)

Spill this subpartition's buffers in a decision.

Parameters:

toSpill - All buffers that need to be spilled belong to this subpartition in a decision.

spillDoneFuture - completed when spill is finished.

Returns:

BufferWithIdentitys about these spill buffers.

releaseSubpartitionBuffers

public void releaseSubpartitionBuffers(List<BufferIndexAndChannel> toRelease)

Release this subpartition's buffers in a decision.

Parameters:

toRelease - All buffers that need to be released belong to this subpartition in a decision.

setOutputMetrics

public void setOutputMetrics(HsOutputMetrics outputMetrics)