org.apache.flink.table.connector.sink

Interface DataStreamSinkProvider

All Superinterfaces:

org.apache.flink.table.connector.sink.DynamicTableSink.SinkRuntimeProvider, org.apache.flink.table.connector.ParallelismProvider

@PublicEvolving
public interface DataStreamSinkProvider
extends org.apache.flink.table.connector.sink.DynamicTableSink.SinkRuntimeProvider, org.apache.flink.table.connector.ParallelismProvider

Provider that consumes a Java DataStream as a runtime implementation for DynamicTableSink.

Note: This provider is only meant for advanced connector developers. Usually, a sink should consist of a single entity expressed via SinkProvider, SinkFunctionProvider, or OutputFormatProvider. When using a DataStream an implementer needs to pay attention to how changes are shuffled to not mess up the changelog per parallel subtask.

Method Summary

Modifier and Type	Method and Description
default org.apache.flink.streaming.api.datastream.DataStreamSink<?>	consumeDataStream(org.apache.flink.streaming.api.datastream.DataStream<org.apache.flink.table.data.RowData> dataStream) Deprecated. Use consumeDataStream(ProviderContext, DataStream) and correctly set a unique identifier for each data stream transformation.
default org.apache.flink.streaming.api.datastream.DataStreamSink<?>	consumeDataStream(ProviderContext providerContext, org.apache.flink.streaming.api.datastream.DataStream<org.apache.flink.table.data.RowData> dataStream) Consumes the given Java DataStream and returns the sink transformation DataStreamSink.
default Optional<Integer>	getParallelism()

Method Detail

consumeDataStream

default org.apache.flink.streaming.api.datastream.DataStreamSink<?> consumeDataStream(ProviderContext providerContext,
                                                                                      org.apache.flink.streaming.api.datastream.DataStream<org.apache.flink.table.data.RowData> dataStream)

Consumes the given Java DataStream and returns the sink transformation DataStreamSink.

Note: If the CompiledPlan feature should be supported, this method MUST set a unique identifier for each transformation/operator in the data stream. This enables stateful Flink version upgrades for streaming jobs. The identifier is used to map state back from a savepoint to an actual operator in the topology. The framework can generate topology-wide unique identifiers with ProviderContext.generateUid(String).

See Also:

SingleOutputStreamOperator.uid(String)

consumeDataStream

@Deprecated
default org.apache.flink.streaming.api.datastream.DataStreamSink<?> consumeDataStream(org.apache.flink.streaming.api.datastream.DataStream<org.apache.flink.table.data.RowData> dataStream)

Deprecated. Use consumeDataStream(ProviderContext, DataStream) and correctly set a unique identifier for each data stream transformation.

Consumes the given Java DataStream and returns the sink transformation DataStreamSink.

getParallelism

default Optional<Integer> getParallelism()

Note: If a custom parallelism is returned and consumeDataStream(ProviderContext, DataStream) applies multiple transformations, make sure to set the same custom parallelism to each operator to not mess up the changelog.

Specified by:

getParallelism in interface org.apache.flink.table.connector.ParallelismProvider