org.apache.flink.runtime.io.network.partition.hybrid

Class HsMemoryDataSpiller

java.lang.Object

org.apache.flink.runtime.io.network.partition.hybrid.HsMemoryDataSpiller

All Implemented Interfaces:

AutoCloseable

public class HsMemoryDataSpiller
extends Object
implements AutoCloseable

This component is responsible for asynchronously writing in-memory data to disk. Each spilling operation will write the disk file sequentially.

Constructor Summary

Constructors

Constructor and Description
HsMemoryDataSpiller(Path dataFilePath)

Method Summary

Modifier and Type	Method and Description
void	close() Close this HsMemoryDataSpiller when resultPartition is closed.
void	release() Release this HsMemoryDataSpiller when resultPartition is released.
CompletableFuture<List<HsFileDataIndex.SpilledBuffer>>	spillAsync(List<BufferWithIdentity> bufferToSpill) Spilling buffers to disk asynchronously.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HsMemoryDataSpiller

public HsMemoryDataSpiller(Path dataFilePath)
                    throws IOException

Throws:

IOException

Method Detail

spillAsync

public CompletableFuture<List<HsFileDataIndex.SpilledBuffer>> spillAsync(List<BufferWithIdentity> bufferToSpill)

Spilling buffers to disk asynchronously.

Parameters:

bufferToSpill - buffers need to be spilled, must ensure that it is sorted by (subpartitionId, bufferIndex).

Returns:

the completable future contains spilled buffers information.

close

public void close()

Close this HsMemoryDataSpiller when resultPartition is closed. It means spiller will no longer accept new spilling operation.

This method only called by main task thread.

Specified by:

close in interface AutoCloseable

release

public void release()

Release this HsMemoryDataSpiller when resultPartition is released. It means spiller will wait for all previous spilling operation done blocking and close the file channel.

This method only called by rpc thread.