TieredStorageMemoryManager (Flink : Runtime 1.18.0 API)

All Known Implementing Classes:

TieredStorageMemoryManagerImpl
```
public interface TieredStorageMemoryManager
```
The TieredStorageMemoryManager is to request or recycle buffers from LocalBufferPool for different memory owners, for example, the tiers, the buffer accumulator, etc. Note that the logic for requesting and recycling buffers is consistent for these owners.
The buffers managed by TieredStorageMemoryManager is categorized into two types: non-reclaimable buffers which cannot be immediately released and reclaimable buffers which can be reclaimed quickly and safely. Non-reclaimable buffers necessitates waiting for other operations to complete before releasing it, such as downstream consumption. On the other hand, reclaimable buffers can be freed up at any time, enabling rapid memory recycling for tasks such as flushing memory to disk or remote storage.
The TieredStorageMemoryManager does not provide strict memory limitations on any user can request. Instead, it only simply provides memory usage hints to memory users. It is very important to note that only users with non-reclaimable should check the memory hints by calling getMaxNonReclaimableBuffers before requesting buffers.

Method Summary

All Methods Instance Methods Abstract Methods
Modifier and Type	Method and Description
`int`	`getMaxNonReclaimableBuffers(Object owner)` Return the number of the non-reclaimable buffers for the owner.
`void`	`listenBufferReclaimRequest(Runnable onBufferReclaimRequest)` Register a listener to listen the buffer reclaim request from the `TieredStorageMemoryManager`.
`int`	`numOwnerRequestedBuffer(Object owner)` Return the number of requested buffers belonging to a specific owner.
`void`	`release()` Release all the resources(if exists) and check the state of the `TieredStorageMemoryManager`.
`BufferBuilder`	`requestBufferBlocking(Object owner)` Request a `BufferBuilder` instance from `BufferPool` for a specific owner.
`void`	`setMetricGroup(TaskIOMetricGroup metricGroup)` Set the `TaskIOMetricGroup` for this memory manager.
`void`	`setup(BufferPool bufferPool, List<TieredStorageMemorySpec> storageMemorySpecs)` Setup the `TieredStorageMemoryManager`.
`void`	`transferBufferOwnership(Object oldOwner, Object newOwner, Buffer buffer)` Notify the memory manager that transferring one buffer's ownership from the old owner to the new owner.

- Method Detail
  - setup
```
void setup(BufferPool bufferPool,
           List<TieredStorageMemorySpec> storageMemorySpecs)
```
    Setup the TieredStorageMemoryManager. When setting up the manager, the TieredStorageMemorySpecs for different tiered storages should be ready to indicate each tiered storage's memory requirement specs.
    
    Parameters:
    
    bufferPool - the local buffer pool
    
    storageMemorySpecs - the memory specs for different tiered storages
  - setMetricGroup
```
void setMetricGroup(TaskIOMetricGroup metricGroup)
```
    Set the TaskIOMetricGroup for this memory manager.
    
    Parameters:
    
    metricGroup - the metric group to set
  - listenBufferReclaimRequest
```
void listenBufferReclaimRequest(Runnable onBufferReclaimRequest)
```
    Register a listener to listen the buffer reclaim request from the TieredStorageMemoryManager.
    When the left buffers in the BufferPool are not enough, TieredStorageMemoryManager will try to reclaim the buffers from the memory owners.
    
    Parameters:
    
    onBufferReclaimRequest - a Runnable to process the buffer reclaim request
  - requestBufferBlocking
```
BufferBuilder requestBufferBlocking(Object owner)
```
    Request a BufferBuilder instance from BufferPool for a specific owner. The TieredStorageMemoryManagerImpl will not check whether a buffer can be requested. The manager only records the number of requested buffers. If the buffers in the BufferPool is not enough, the manager will request each tiered storage to reclaim their requested buffers as much as possible.
    This is not thread safe and is expected to be called only from the task thread.
    
    Parameters:
    
    owner - the owner to request buffer
    
    Returns:
    
    the requested buffer
  - getMaxNonReclaimableBuffers
```
int getMaxNonReclaimableBuffers(Object owner)
```
    Return the number of the non-reclaimable buffers for the owner.
    Note that the available buffers are calculated dynamically based on some conditions, for example, the state of the BufferPool, the TieredStorageMemorySpec of the owner, etc. So the caller should always check before requesting non-reclaimable buffers.
    When invoking this method, the caller should be aware that the return value may occasionally be negative. This is due to the possibility of the buffer pool size shrinking to a point where it is smaller than the buffers owned by other users. In such cases, the maximum non-reclaimable buffer value returned may be negative.
  - numOwnerRequestedBuffer
```
int numOwnerRequestedBuffer(Object owner)
```
    Return the number of requested buffers belonging to a specific owner.
    
    Parameters:
    
    owner - the owner of requesting buffers
    
    Returns:
    
    the number of requested buffers belonging to the owner.
  - transferBufferOwnership
```
void transferBufferOwnership(Object oldOwner,
                             Object newOwner,
                             Buffer buffer)
```
    Notify the memory manager that transferring one buffer's ownership from the old owner to the new owner.
    
    Parameters:
    
    oldOwner - the old owner of one buffer
    
    newOwner - the new owner of one buffer
    
    buffer - the buffer to transfer the ownership
  - release
```
void release()
```
    Release all the resources(if exists) and check the state of the TieredStorageMemoryManager.

Interface TieredStorageMemoryManager

Method Summary

Method Detail

setup

setMetricGroup

listenBufferReclaimRequest

requestBufferBlocking

getMaxNonReclaimableBuffers

numOwnerRequestedBuffer

transferBufferOwnership

release