org.apache.flink.streaming.api.graph

类 StreamGraphGenerator

java.lang.Object

org.apache.flink.streaming.api.graph.StreamGraphGenerator

@Internal
public class StreamGraphGenerator
extends Object

A generator that generates a StreamGraph from a graph of Transformations.

This traverses the tree of Transformations starting from the sinks. At each transformation we recursively transform the inputs, then create a node in the StreamGraph and add edges from the input Nodes to our newly created node. The transformation methods return the IDs of the nodes in the StreamGraph that represent the input transformation. Several IDs can be returned to be able to deal with feedback transformations and unions.

Partitioning, split/select and union don't create actual nodes in the StreamGraph. For these, we create a virtual node in the StreamGraph that holds the specific property, i.e. partitioning, selector and so on. When an edge is created from a virtual node to a downstream node the StreamGraph resolved the id of the original node and creates an edge in the graph with the desired property. For example, if you have this graph:

     Map-1 -> HashPartition-2 -> Map-3
 

where the numbers represent transformation IDs. We first recurse all the way down. Map-1 is transformed, i.e. we create a StreamNode with ID 1. Then we transform the HashPartition, for this, we create virtual node of ID 4 that holds the property HashPartition. This transformation returns the ID 4. Then we transform the Map-3. We add the edge 4 -> 3. The StreamGraph resolved the actual node with ID 1 and creates and edge 1 -> 3 with the property HashPartition.

字段概要

字段

限定符和类型	字段和说明
static String	DEFAULT_BATCH_JOB_NAME
static int	DEFAULT_LOWER_BOUND_MAX_PARALLELISM
static String	DEFAULT_SLOT_SHARING_GROUP
static String	DEFAULT_STREAMING_JOB_NAME
static TimeCharacteristic	DEFAULT_TIME_CHARACTERISTIC
protected static Integer	iterationIdCounter

构造器概要

构造器

构造器和说明
StreamGraphGenerator(List<org.apache.flink.api.dag.Transformation<?>> transformations, org.apache.flink.api.common.ExecutionConfig executionConfig, CheckpointConfig checkpointConfig)
StreamGraphGenerator(List<org.apache.flink.api.dag.Transformation<?>> transformations, org.apache.flink.api.common.ExecutionConfig executionConfig, CheckpointConfig checkpointConfig, org.apache.flink.configuration.ReadableConfig configuration)

方法概要

限定符和类型	方法和说明
StreamGraph	generate()
static int	getNewIterationNodeId()
StreamGraphGenerator	setChaining(boolean chaining)
StreamGraphGenerator	setChangelogStateBackendEnabled(org.apache.flink.util.TernaryBoolean changelogStateBackendEnabled)
StreamGraphGenerator	setDefaultBufferTimeout(long defaultBufferTimeout)
StreamGraphGenerator	setSavepointDir(org.apache.flink.core.fs.Path savepointDir)
void	setSavepointRestoreSettings(org.apache.flink.runtime.jobgraph.SavepointRestoreSettings savepointRestoreSettings)
StreamGraphGenerator	setSlotSharingGroupResource(Map<String,org.apache.flink.runtime.clusterframework.types.ResourceProfile> slotSharingGroupResources) Specify fine-grained resource requirements for slot sharing groups.
StreamGraphGenerator	setStateBackend(org.apache.flink.runtime.state.StateBackend stateBackend)
StreamGraphGenerator	setTimeCharacteristic(TimeCharacteristic timeCharacteristic)
StreamGraphGenerator	setUserArtifacts(Collection<org.apache.flink.api.java.tuple.Tuple2<String,org.apache.flink.api.common.cache.DistributedCache.DistributedCacheEntry>> userArtifacts)

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

字段详细资料

DEFAULT_LOWER_BOUND_MAX_PARALLELISM

public static final int DEFAULT_LOWER_BOUND_MAX_PARALLELISM

另请参阅:

常量字段值

DEFAULT_TIME_CHARACTERISTIC

public static final TimeCharacteristic DEFAULT_TIME_CHARACTERISTIC

DEFAULT_STREAMING_JOB_NAME

public static final String DEFAULT_STREAMING_JOB_NAME

另请参阅:

常量字段值

DEFAULT_BATCH_JOB_NAME

public static final String DEFAULT_BATCH_JOB_NAME

另请参阅:

常量字段值

DEFAULT_SLOT_SHARING_GROUP

public static final String DEFAULT_SLOT_SHARING_GROUP

另请参阅:

常量字段值

iterationIdCounter

protected static Integer iterationIdCounter

构造器详细资料

StreamGraphGenerator

public StreamGraphGenerator(List<org.apache.flink.api.dag.Transformation<?>> transformations,
                            org.apache.flink.api.common.ExecutionConfig executionConfig,
                            CheckpointConfig checkpointConfig)

StreamGraphGenerator

public StreamGraphGenerator(List<org.apache.flink.api.dag.Transformation<?>> transformations,
                            org.apache.flink.api.common.ExecutionConfig executionConfig,
                            CheckpointConfig checkpointConfig,
                            org.apache.flink.configuration.ReadableConfig configuration)

方法详细资料

getNewIterationNodeId

public static int getNewIterationNodeId()

setSavepointDir

public StreamGraphGenerator setSavepointDir(org.apache.flink.core.fs.Path savepointDir)

setStateBackend

public StreamGraphGenerator setStateBackend(org.apache.flink.runtime.state.StateBackend stateBackend)

setChangelogStateBackendEnabled

public StreamGraphGenerator setChangelogStateBackendEnabled(org.apache.flink.util.TernaryBoolean changelogStateBackendEnabled)

setChaining

public StreamGraphGenerator setChaining(boolean chaining)

setUserArtifacts

public StreamGraphGenerator setUserArtifacts(Collection<org.apache.flink.api.java.tuple.Tuple2<String,org.apache.flink.api.common.cache.DistributedCache.DistributedCacheEntry>> userArtifacts)

setTimeCharacteristic

public StreamGraphGenerator setTimeCharacteristic(TimeCharacteristic timeCharacteristic)

setDefaultBufferTimeout

public StreamGraphGenerator setDefaultBufferTimeout(long defaultBufferTimeout)

setSlotSharingGroupResource

public StreamGraphGenerator setSlotSharingGroupResource(Map<String,org.apache.flink.runtime.clusterframework.types.ResourceProfile> slotSharingGroupResources)

Specify fine-grained resource requirements for slot sharing groups.

Note that a slot sharing group hints the scheduler that the grouped operators CAN be deployed into a shared slot. There's no guarantee that the scheduler always deploy the grouped operators together. In cases grouped operators are deployed into separate slots, the slot resources will be derived from the specified group requirements.

setSavepointRestoreSettings

public void setSavepointRestoreSettings(org.apache.flink.runtime.jobgraph.SavepointRestoreSettings savepointRestoreSettings)

generate

public StreamGraph generate()