org.apache.flink.runtime.io.network.api.writer

枚举 SubtaskStateMapper

java.lang.Object

java.lang.Enum<SubtaskStateMapper>

org.apache.flink.runtime.io.network.api.writer.SubtaskStateMapper

所有已实现的接口:

Serializable, Comparable<SubtaskStateMapper>

@Internal
public enum SubtaskStateMapper
extends Enum<SubtaskStateMapper>

The SubtaskStateMapper narrows down the subtasks that need to be read during rescaling to recover from a particular subtask when in-flight data has been stored in the checkpoint.

Mappings of old subtasks to new subtasks may be unique or non-unique. A unique assignment means that a particular old subtask is only assigned to exactly one new subtask. Non-unique assignments require filtering downstream. That means that the receiver side has to cross-verify for a deserialized record if it truly belongs to the new subtask or not. Most SubtaskStateMapper will only produce unique assignments and are thus optimal. Some rescaler, such as RANGE, create a mixture of unique and non-unique mappings, where downstream tasks need to filter on some mapped subtasks.

枚举常量概要

枚举常量

枚举常量和说明
ARBITRARY Extra state is redistributed to other subtasks without any specific guarantee (only that up- and downstream are matched).
FIRST Restores extra subtasks to the first subtask.
FULL Replicates the state to all subtasks.
RANGE Remaps old ranges to new ranges.
ROUND_ROBIN Redistributes subtask state in a round robin fashion.
UNSUPPORTED

方法概要

限定符和类型	方法和说明
RescaleMappings	getNewToOldSubtasksMapping(int oldParallelism, int newParallelism) Returns a mapping new subtask index to all old subtask indexes.
abstract int[]	getOldSubtasks(int newSubtaskIndex, int oldNumberOfSubtasks, int newNumberOfSubtasks) Returns all old subtask indexes that need to be read to restore all buffers for the given new subtask index on rescale.
boolean	isAmbiguous() Returns true iff this mapper can potentially lead to ambiguous mappings where the different new subtasks map to the same old subtask.
static SubtaskStateMapper	valueOf(String name) 返回带有指定名称的该类型的枚举常量。
static SubtaskStateMapper[]	values() 按照声明该枚举类型的常量的顺序, 返回 包含这些常量的数组。

从类继承的方法 java.lang.Enum

clone, compareTo, equals, finalize, getDeclaringClass, hashCode, name, ordinal, toString, valueOf

从类继承的方法 java.lang.Object

getClass, notify, notifyAll, wait, wait, wait

枚举常量详细资料

ARBITRARY

public static final SubtaskStateMapper ARBITRARY

Extra state is redistributed to other subtasks without any specific guarantee (only that up- and downstream are matched).

FIRST

public static final SubtaskStateMapper FIRST

Restores extra subtasks to the first subtask.

FULL

public static final SubtaskStateMapper FULL

Replicates the state to all subtasks. This rescaling causes a huge overhead and completely relies on filtering the data downstream.

This strategy should only be used as a fallback.

RANGE

public static final SubtaskStateMapper RANGE

Remaps old ranges to new ranges. For minor rescaling that means that new subtasks are mostly assigned 2 old subtasks.

Example:
old assignment: 0 -> [0;43); 1 -> [43;87); 2 -> [87;128)
new assignment: 0 -> [0;64]; 1 -> [64;128)
subtask 0 recovers data from old subtask 0 + 1 and subtask 1 recovers data from old subtask 1 + 2

For all downscale from n to [n-1 .. n/2], each new subtasks get exactly two old subtasks assigned.

For all upscale from n to [n+1 .. 2*n-1], most subtasks get two old subtasks assigned, except the two outermost.

Larger scale factors (<n/2, >2*n), will increase the number of old subtasks accordingly. However, they will also create more unique assignment, where an old subtask is exclusively assigned to a new subtask. Thus, the number of non-unique mappings is upper bound by 2*n.

ROUND_ROBIN

public static final SubtaskStateMapper ROUND_ROBIN

Redistributes subtask state in a round robin fashion. Returns a mapping of newIndex -> oldIndexes. The mapping is accessed by using Bitset oldIndexes = mapping.get(newIndex).

For oldParallelism < newParallelism, that mapping is trivial. For example if oldParallelism = 6 and newParallelism = 10.

New index	Old indexes
0	0
1	1
...
5	5
6
...
9

For oldParallelism > newParallelism, new indexes get multiple assignments by wrapping around assignments in a round-robin fashion. For example if oldParallelism = 10 and newParallelism = 4.

New index	Old indexes
0	0, 4, 8
1	1, 5, 9
2	2, 6
3	3, 7

UNSUPPORTED

public static final SubtaskStateMapper UNSUPPORTED

方法详细资料

values

public static SubtaskStateMapper[] values()

按照声明该枚举类型的常量的顺序, 返回 包含这些常量的数组。该方法可用于迭代 常量, 如下所示:

for (SubtaskStateMapper c : SubtaskStateMapper.values())
    System.out.println(c);

返回:

按照声明该枚举类型的常量的顺序返回的包含这些常量的数组

valueOf

public static SubtaskStateMapper valueOf(String name)

返回带有指定名称的该类型的枚举常量。 字符串必须与用于声明该类型的枚举常量的 标识符完全匹配。(不允许有多余 的空格字符。)

参数:

name - 要返回的枚举常量的名称。

返回:

返回带有指定名称的枚举常量

抛出:

IllegalArgumentException - 如果该枚举类型没有带有指定名称的常量

NullPointerException - 如果参数为空值

getOldSubtasks

public abstract int[] getOldSubtasks(int newSubtaskIndex,
                                     int oldNumberOfSubtasks,
                                     int newNumberOfSubtasks)

Returns all old subtask indexes that need to be read to restore all buffers for the given new subtask index on rescale.

getNewToOldSubtasksMapping

public RescaleMappings getNewToOldSubtasksMapping(int oldParallelism,
                                                  int newParallelism)

Returns a mapping new subtask index to all old subtask indexes.

isAmbiguous

public boolean isAmbiguous()

Returns true iff this mapper can potentially lead to ambiguous mappings where the different new subtasks map to the same old subtask. The assumption is that such replicated data needs to be filtered.