org.apache.flink.runtime.io.network.partition

类 SortBasedDataBuffer

java.lang.Object

org.apache.flink.runtime.io.network.partition.SortBasedDataBuffer

所有已实现的接口:

DataBuffer

@NotThreadSafe
public class SortBasedDataBuffer
extends Object
implements DataBuffer

A DataBuffer implementation which sorts all appended records only by subpartition index. Records of the same subpartition keep the appended order.

It maintains a list of MemorySegments as a joint buffer. Data will be appended to the joint buffer sequentially. When writing a record, an index entry will be appended first. An index entry consists of 4 fields: 4 bytes for record length, 4 bytes for Buffer.DataType and 8 bytes for address pointing to the next index entry of the same channel which will be used to index the next record to read when coping data from this DataBuffer. For simplicity, no index entry can span multiple segments. The corresponding record data is seated right after its index entry and different from the index entry, records have variable length thus may span multiple segments.

构造器概要

构造器

构造器和说明
SortBasedDataBuffer(LinkedList<org.apache.flink.core.memory.MemorySegment> freeSegments, BufferRecycler bufferRecycler, int numSubpartitions, int bufferSize, int numGuaranteedBuffers, int[] customReadOrder)

方法概要

限定符和类型	方法和说明
boolean	append(ByteBuffer source, int targetChannel, Buffer.DataType dataType) No partial record will be written to this SortBasedDataBuffer, which means that either all data of target record will be written or nothing will be written.
void	finish() Finishes this DataBuffer which means no record can be appended any more.
BufferWithChannel	getNextBuffer(org.apache.flink.core.memory.MemorySegment transitBuffer) Copies data in this DataBuffer to the target MemorySegment in channel index order and returns BufferWithChannel which contains the copied data and the corresponding channel index.
boolean	hasRemaining() Returns true if not all data appended to this DataBuffer is consumed.
boolean	isFinished() Whether this DataBuffer is finished or not.
boolean	isReleased() Whether this DataBuffer is released or not.
long	numTotalBytes() Returns the total number of bytes written to this DataBuffer.
long	numTotalRecords() Returns the total number of records written to this DataBuffer.
void	release() Releases this DataBuffer which releases all resources.

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

构造器详细资料

SortBasedDataBuffer

public SortBasedDataBuffer(LinkedList<org.apache.flink.core.memory.MemorySegment> freeSegments,
                           BufferRecycler bufferRecycler,
                           int numSubpartitions,
                           int bufferSize,
                           int numGuaranteedBuffers,
                           @Nullable
                           int[] customReadOrder)

方法详细资料

append

public boolean append(ByteBuffer source,
                      int targetChannel,
                      Buffer.DataType dataType)
               throws IOException

No partial record will be written to this SortBasedDataBuffer, which means that either all data of target record will be written or nothing will be written.

指定者:

append 在接口中 DataBuffer

抛出:

IOException

getNextBuffer

public BufferWithChannel getNextBuffer(org.apache.flink.core.memory.MemorySegment transitBuffer)

从接口复制的说明: DataBuffer

Copies data in this DataBuffer to the target MemorySegment in channel index order and returns BufferWithChannel which contains the copied data and the corresponding channel index.

指定者:

getNextBuffer 在接口中 DataBuffer

numTotalRecords

public long numTotalRecords()

从接口复制的说明: DataBuffer

Returns the total number of records written to this DataBuffer.

指定者:

numTotalRecords 在接口中 DataBuffer

numTotalBytes

public long numTotalBytes()

从接口复制的说明: DataBuffer

Returns the total number of bytes written to this DataBuffer.

指定者:

numTotalBytes 在接口中 DataBuffer

hasRemaining

public boolean hasRemaining()

从接口复制的说明: DataBuffer

Returns true if not all data appended to this DataBuffer is consumed.

指定者:

hasRemaining 在接口中 DataBuffer

finish

public void finish()

从接口复制的说明: DataBuffer

Finishes this DataBuffer which means no record can be appended any more.

指定者:

finish 在接口中 DataBuffer

isFinished

public boolean isFinished()

从接口复制的说明: DataBuffer

Whether this DataBuffer is finished or not.

指定者:

isFinished 在接口中 DataBuffer

release

public void release()

从接口复制的说明: DataBuffer

Releases this DataBuffer which releases all resources.

指定者:

release 在接口中 DataBuffer

isReleased

public boolean isReleased()

从接口复制的说明: DataBuffer

Whether this DataBuffer is released or not.

指定者:

isReleased 在接口中 DataBuffer