Package org.apache.beam.runners.dataflow

Class DataflowPipelineTranslator

java.lang.Object

org.apache.beam.runners.dataflow.DataflowPipelineTranslator

public class DataflowPipelineTranslator
extends java.lang.Object

DataflowPipelineTranslator knows how to translate Pipeline objects into Cloud Dataflow Service API Jobs.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	DataflowPipelineTranslator.JobSpecification	The result of a job translation.

Method Summary

Modifier and Type	Method	Description
static DataflowPipelineTranslator	fromOptions(DataflowPipelineOptions options)	Constructs a translator from the provided options.
<TransformT extends org.apache.beam.sdk.transforms.PTransform> TransformTranslator<TransformT>	getTransformTranslator(java.lang.Class<TransformT> transformClass)	Returns the TransformTranslator to use for instances of the specified PTransform class, or null if none registered.
static java.lang.String	jobToString(com.google.api.services.dataflow.model.Job job)	Renders a Job as a string.
static <TransformT extends org.apache.beam.sdk.transforms.PTransform> void	registerTransformTranslator(java.lang.Class<TransformT> transformClass, TransformTranslator<? extends TransformT> transformTranslator)	Records that instances of the specified PTransform class should be translated by default by the corresponding TransformTranslator.
java.lang.String	toString()
DataflowPipelineTranslator.JobSpecification	translate(org.apache.beam.sdk.Pipeline pipeline, org.apache.beam.model.pipeline.v1.RunnerApi.Pipeline pipelineProto, org.apache.beam.sdk.util.construction.SdkComponents sdkComponents, DataflowRunner runner, java.util.List<com.google.api.services.dataflow.model.DataflowPackage> packages)	Translates a Pipeline into a JobSpecification.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Detail

fromOptions

public static DataflowPipelineTranslator fromOptions(DataflowPipelineOptions options)

Constructs a translator from the provided options.

Parameters:

options - Properties that configure the translator.

Returns:

The newly created translator.

translate

public DataflowPipelineTranslator.JobSpecification translate(org.apache.beam.sdk.Pipeline pipeline,
                                                             org.apache.beam.model.pipeline.v1.RunnerApi.Pipeline pipelineProto,
                                                             org.apache.beam.sdk.util.construction.SdkComponents sdkComponents,
                                                             DataflowRunner runner,
                                                             java.util.List<com.google.api.services.dataflow.model.DataflowPackage> packages)

Translates a Pipeline into a JobSpecification.

jobToString

public static java.lang.String jobToString(com.google.api.services.dataflow.model.Job job)

Renders a Job as a string.

registerTransformTranslator

public static <TransformT extends org.apache.beam.sdk.transforms.PTransform> void registerTransformTranslator(java.lang.Class<TransformT> transformClass,
                                                                                                              TransformTranslator<? extends TransformT> transformTranslator)

Records that instances of the specified PTransform class should be translated by default by the corresponding TransformTranslator.

getTransformTranslator

public <TransformT extends org.apache.beam.sdk.transforms.PTransform> TransformTranslator<TransformT> getTransformTranslator(java.lang.Class<TransformT> transformClass)

Returns the TransformTranslator to use for instances of the specified PTransform class, or null if none registered.

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object