Package org.apache.beam.runners.dataflow.options

Provides PipelineOptions specific to Google Cloud Dataflow.

Interface Summary

Interface	Description
DataflowPipelineDebugOptions	Internal.
DataflowPipelineOptions	Options that can be used to configure the DataflowRunner.
DataflowPipelineWorkerPoolOptions	Options that are used to configure the Dataflow pipeline worker pool.
DataflowProfilingOptions	Options for controlling profiling of pipeline execution.
DataflowStreamingPipelineOptions	[Internal] Options for configuring StreamingDataflowWorker.
DataflowWorkerHarnessOptions	Options that are used exclusively within the Dataflow worker harness.
DataflowWorkerLoggingOptions	Deprecated. This interface will no longer be the source of truth for worker logging configuration once jobs are executed using a dedicated SDK harness instead of user code being co-located alongside Dataflow worker code.

Class Summary

Class	Description
DataflowPipelineDebugOptions.DataflowClientFactory	Returns the default Dataflow client built from the passed in PipelineOptions.
DataflowPipelineDebugOptions.StagerFactory	Creates a Stager object using the class specified in DataflowPipelineDebugOptions.getStagerClass().
DataflowPipelineDebugOptions.UnboundedReaderMaxReadTimeFactory	Sets Integer value based on old, deprecated field (DataflowPipelineDebugOptions.getUnboundedReaderMaxReadTimeSec()).
DataflowPipelineOptions.StagingLocationFactory	Returns a default staging location under GcpOptions.getGcpTempLocation().
DataflowProfilingOptions.DataflowProfilingAgentConfiguration	Configuration the for profiling agent.
DataflowStreamingPipelineOptions.EnableWindmillServiceDirectPathFactory	EnableStreamingEngine defaults to false unless one of the experiment is set.
DataflowStreamingPipelineOptions.GlobalConfigRefreshPeriodFactory	Read global get config request period from system property 'windmill.global_config_refresh_period'.
DataflowStreamingPipelineOptions.HarnessUpdateReportingPeriodFactory	Read counter reporting period from system property 'windmill.harness_update_reporting_period'.
DataflowStreamingPipelineOptions.LocalWindmillHostportFactory	Factory for creating local Windmill address.
DataflowStreamingPipelineOptions.MaxStackTraceDepthToReportFactory	Read 'MaxStackTraceToReport' from system property 'windmill.max_stack_trace_to_report' or Integer.MAX_VALUE if unspecified.
DataflowStreamingPipelineOptions.PeriodicStatusPageDirectoryFactory	Read 'PeriodicStatusPageOutputDirector' from system property 'windmill.periodic_status_page_directory' or null if unspecified.
DataflowStreamingPipelineOptions.WindmillServiceStreamingRpcBatchLimitFactory	Factory for setting value of WindmillServiceStreamingRpcBatchLimit based on environment.
DataflowWorkerLoggingOptions.WorkerLogLevelOverrides	Defines a log level override for a specific class, package, or name.
DefaultGcpRegionFactory	Factory for a default value for Google Cloud region according to https://cloud.google.com/compute/docs/gcloud-compute/#default-properties.

Enum Summary

Enum	Description
DataflowPipelineOptions.FlexResourceSchedulingGoal	Set of available Flexible Resource Scheduling goals.
DataflowPipelineWorkerPoolOptions.AutoscalingAlgorithmType	Type of autoscaling algorithm to use.
DataflowWorkerLoggingOptions.Level	The set of log levels that can be used on the Dataflow worker.