com.google.api.services.dataflow.model

Class Environment

java.lang.Object

java.util.AbstractMap<String,Object>

com.google.api.client.util.GenericData

com.google.api.client.json.GenericJson

com.google.api.services.dataflow.model.Environment

All Implemented Interfaces:

Cloneable, Map<String,Object>

public final class Environment
extends com.google.api.client.json.GenericJson

Describes the environment in which a Dataflow Job runs.

This is the Java data model class that specifies how to parse/serialize into the JSON that is transmitted over HTTP when working with the Dataflow API. For a detailed explanation see: https://developers.google.com/api-client-library/java/google-http-java-client/json

Author:

Google, Inc.

Nested Class Summary

Nested classes/interfaces inherited from class com.google.api.client.util.GenericData

com.google.api.client.util.GenericData.Flags

Nested classes/interfaces inherited from class java.util.AbstractMap

AbstractMap.SimpleEntry<K,V>, AbstractMap.SimpleImmutableEntry<K,V>

Nested classes/interfaces inherited from interface java.util.Map

Map.Entry<K,V>

Constructor Summary

Constructors

Constructor and Description
Environment()

Method Summary

Modifier and Type	Method and Description
Environment	clone()
String	getClusterManagerApiService() The type of cluster manager API to use.
String	getDataset() Optional.
DebugOptions	getDebugOptions() Optional.
List<String>	getExperiments() The list of experiments to enable.
String	getFlexResourceSchedulingGoal() Optional.
Map<String,Object>	getInternalExperiments() Experimental settings.
Map<String,Object>	getSdkPipelineOptions() The Cloud Dataflow SDK pipeline options specified by the user.
String	getServiceAccountEmail() Optional.
String	getServiceKmsKeyName() Optional.
List<String>	getServiceOptions() Optional.
String	getShuffleMode() Output only.
String	getStreamingMode() Optional.
String	getTempStoragePrefix() The prefix of the resources the system should use for temporary storage.
Boolean	getUsePublicIps() Optional.
Map<String,Object>	getUserAgent() A description of the process that generated the request.
Boolean	getUseStreamingEngineResourceBasedBilling() Output only.
Map<String,Object>	getVersion() A structure describing which components and their versions of the service are required in order to run the job.
List<WorkerPool>	getWorkerPools() The worker pools.
String	getWorkerRegion() Optional.
String	getWorkerZone() Optional.
Environment	set(String fieldName, Object value)
Environment	setClusterManagerApiService(String clusterManagerApiService) The type of cluster manager API to use.
Environment	setDataset(String dataset) Optional.
Environment	setDebugOptions(DebugOptions debugOptions) Optional.
Environment	setExperiments(List<String> experiments) The list of experiments to enable.
Environment	setFlexResourceSchedulingGoal(String flexResourceSchedulingGoal) Optional.
Environment	setInternalExperiments(Map<String,Object> internalExperiments) Experimental settings.
Environment	setSdkPipelineOptions(Map<String,Object> sdkPipelineOptions) The Cloud Dataflow SDK pipeline options specified by the user.
Environment	setServiceAccountEmail(String serviceAccountEmail) Optional.
Environment	setServiceKmsKeyName(String serviceKmsKeyName) Optional.
Environment	setServiceOptions(List<String> serviceOptions) Optional.
Environment	setShuffleMode(String shuffleMode) Output only.
Environment	setStreamingMode(String streamingMode) Optional.
Environment	setTempStoragePrefix(String tempStoragePrefix) The prefix of the resources the system should use for temporary storage.
Environment	setUsePublicIps(Boolean usePublicIps) Optional.
Environment	setUserAgent(Map<String,Object> userAgent) A description of the process that generated the request.
Environment	setUseStreamingEngineResourceBasedBilling(Boolean useStreamingEngineResourceBasedBilling) Output only.
Environment	setVersion(Map<String,Object> version) A structure describing which components and their versions of the service are required in order to run the job.
Environment	setWorkerPools(List<WorkerPool> workerPools) The worker pools.
Environment	setWorkerRegion(String workerRegion) Optional.
Environment	setWorkerZone(String workerZone) Optional.

Methods inherited from class com.google.api.client.json.GenericJson

getFactory, setFactory, toPrettyString, toString

Methods inherited from class com.google.api.client.util.GenericData

entrySet, equals, get, getClassInfo, getUnknownKeys, hashCode, put, putAll, remove, setUnknownKeys

Methods inherited from class java.util.AbstractMap

clear, containsKey, containsValue, isEmpty, keySet, size, values

Methods inherited from class java.lang.Object

finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface java.util.Map

compute, computeIfAbsent, computeIfPresent, forEach, getOrDefault, merge, putIfAbsent, remove, replace, replace, replaceAll

Constructor Detail

Environment

public Environment()

Method Detail

getClusterManagerApiService

public String getClusterManagerApiService()

The type of cluster manager API to use. If unknown or unspecified, the service will attempt to choose a reasonable default. This should be in the form of the API service name, e.g. "compute.googleapis.com".

Returns:

value or null for none

setClusterManagerApiService

public Environment setClusterManagerApiService(String clusterManagerApiService)

The type of cluster manager API to use. If unknown or unspecified, the service will attempt to choose a reasonable default. This should be in the form of the API service name, e.g. "compute.googleapis.com".

Parameters:

clusterManagerApiService - clusterManagerApiService or null for none

getDataset

public String getDataset()

Optional. The dataset for the current project where various workflow related tables are stored. The supported resource type is: Google BigQuery: bigquery.googleapis.com/{dataset}

Returns:

value or null for none

setDataset

public Environment setDataset(String dataset)

Optional. The dataset for the current project where various workflow related tables are stored. The supported resource type is: Google BigQuery: bigquery.googleapis.com/{dataset}

Parameters:

dataset - dataset or null for none

getDebugOptions

public DebugOptions getDebugOptions()

Optional. Any debugging options to be supplied to the job.

Returns:

value or null for none

setDebugOptions

public Environment setDebugOptions(DebugOptions debugOptions)

Optional. Any debugging options to be supplied to the job.

Parameters:

debugOptions - debugOptions or null for none

getExperiments

public List<String> getExperiments()

The list of experiments to enable. This field should be used for SDK related experiments and not for service related experiments. The proper field for service related experiments is service_options.

Returns:

value or null for none

setExperiments

public Environment setExperiments(List<String> experiments)

The list of experiments to enable. This field should be used for SDK related experiments and not for service related experiments. The proper field for service related experiments is service_options.

Parameters:

experiments - experiments or null for none

getFlexResourceSchedulingGoal

public String getFlexResourceSchedulingGoal()

Optional. Which Flexible Resource Scheduling mode to run in.

Returns:

value or null for none

setFlexResourceSchedulingGoal

public Environment setFlexResourceSchedulingGoal(String flexResourceSchedulingGoal)

Optional. Which Flexible Resource Scheduling mode to run in.

Parameters:

flexResourceSchedulingGoal - flexResourceSchedulingGoal or null for none

getInternalExperiments

public Map<String,Object> getInternalExperiments()

Experimental settings.

Returns:

value or null for none

setInternalExperiments

public Environment setInternalExperiments(Map<String,Object> internalExperiments)

Experimental settings.

Parameters:

internalExperiments - internalExperiments or null for none

getSdkPipelineOptions

public Map<String,Object> getSdkPipelineOptions()

The Cloud Dataflow SDK pipeline options specified by the user. These options are passed through the service and are used to recreate the SDK pipeline options on the worker in a language agnostic and platform independent way.

Returns:

value or null for none

setSdkPipelineOptions

public Environment setSdkPipelineOptions(Map<String,Object> sdkPipelineOptions)

The Cloud Dataflow SDK pipeline options specified by the user. These options are passed through the service and are used to recreate the SDK pipeline options on the worker in a language agnostic and platform independent way.

Parameters:

sdkPipelineOptions - sdkPipelineOptions or null for none

getServiceAccountEmail

public String getServiceAccountEmail()

Optional. Identity to run virtual machines as. Defaults to the default account.

Returns:

value or null for none

setServiceAccountEmail

public Environment setServiceAccountEmail(String serviceAccountEmail)

Optional. Identity to run virtual machines as. Defaults to the default account.

Parameters:

serviceAccountEmail - serviceAccountEmail or null for none

getServiceKmsKeyName

public String getServiceKmsKeyName()

Optional. If set, contains the Cloud KMS key identifier used to encrypt data at rest, AKA a Customer Managed Encryption Key (CMEK). Format: projects/PROJECT_ID/locations/LOCATION/keyRings/KEY_RING/cryptoKeys/KEY

Returns:

value or null for none

setServiceKmsKeyName

public Environment setServiceKmsKeyName(String serviceKmsKeyName)

Optional. If set, contains the Cloud KMS key identifier used to encrypt data at rest, AKA a Customer Managed Encryption Key (CMEK). Format: projects/PROJECT_ID/locations/LOCATION/keyRings/KEY_RING/cryptoKeys/KEY

Parameters:

serviceKmsKeyName - serviceKmsKeyName or null for none

getServiceOptions

public List<String> getServiceOptions()

Optional. The list of service options to enable. This field should be used for service related experiments only. These experiments, when graduating to GA, should be replaced by dedicated fields or become default (i.e. always on).

Returns:

value or null for none

setServiceOptions

public Environment setServiceOptions(List<String> serviceOptions)

Optional. The list of service options to enable. This field should be used for service related experiments only. These experiments, when graduating to GA, should be replaced by dedicated fields or become default (i.e. always on).

Parameters:

serviceOptions - serviceOptions or null for none

getShuffleMode

public String getShuffleMode()

Output only. The shuffle mode used for the job.

Returns:

value or null for none

setShuffleMode

public Environment setShuffleMode(String shuffleMode)

Output only. The shuffle mode used for the job.

Parameters:

shuffleMode - shuffleMode or null for none

getStreamingMode

public String getStreamingMode()

Optional. Specifies the Streaming Engine message processing guarantees. Reduces cost and latency but might result in duplicate messages committed to storage. Designed to run simple mapping streaming ETL jobs at the lowest cost. For example, Change Data Capture (CDC) to BigQuery is a canonical use case. For more information, see [Set the pipeline streaming mode](https://cloud.google.com/dataflow/docs/guides/streaming-modes).

Returns:

value or null for none

setStreamingMode

public Environment setStreamingMode(String streamingMode)

Optional. Specifies the Streaming Engine message processing guarantees. Reduces cost and latency but might result in duplicate messages committed to storage. Designed to run simple mapping streaming ETL jobs at the lowest cost. For example, Change Data Capture (CDC) to BigQuery is a canonical use case. For more information, see [Set the pipeline streaming mode](https://cloud.google.com/dataflow/docs/guides/streaming-modes).

Parameters:

streamingMode - streamingMode or null for none

getTempStoragePrefix

public String getTempStoragePrefix()

The prefix of the resources the system should use for temporary storage. The system will append the suffix "/temp-{JOBNAME} to this resource prefix, where {JOBNAME} is the value of the job_name field. The resulting bucket and object prefix is used as the prefix of the resources used to store temporary data needed during the job execution. NOTE: This will override the value in taskrunner_settings. The supported resource type is: Google Cloud Storage: storage.googleapis.com/{bucket}/{object} bucket.storage.googleapis.com/{object}

Returns:

value or null for none

setTempStoragePrefix

public Environment setTempStoragePrefix(String tempStoragePrefix)

The prefix of the resources the system should use for temporary storage. The system will append the suffix "/temp-{JOBNAME} to this resource prefix, where {JOBNAME} is the value of the job_name field. The resulting bucket and object prefix is used as the prefix of the resources used to store temporary data needed during the job execution. NOTE: This will override the value in taskrunner_settings. The supported resource type is: Google Cloud Storage: storage.googleapis.com/{bucket}/{object} bucket.storage.googleapis.com/{object}

Parameters:

tempStoragePrefix - tempStoragePrefix or null for none

getUsePublicIps

public Boolean getUsePublicIps()

Optional. True when any worker pool that uses public IPs is present.

Returns:

value or null for none

setUsePublicIps

public Environment setUsePublicIps(Boolean usePublicIps)

Optional. True when any worker pool that uses public IPs is present.

Parameters:

usePublicIps - usePublicIps or null for none

getUseStreamingEngineResourceBasedBilling

public Boolean getUseStreamingEngineResourceBasedBilling()

Output only. Whether the job uses the Streaming Engine resource-based billing model.

Returns:

value or null for none

setUseStreamingEngineResourceBasedBilling

public Environment setUseStreamingEngineResourceBasedBilling(Boolean useStreamingEngineResourceBasedBilling)

Output only. Whether the job uses the Streaming Engine resource-based billing model.

Parameters:

useStreamingEngineResourceBasedBilling - useStreamingEngineResourceBasedBilling or null for none

getUserAgent

public Map<String,Object> getUserAgent()

A description of the process that generated the request.

Returns:

value or null for none

setUserAgent

public Environment setUserAgent(Map<String,Object> userAgent)

A description of the process that generated the request.

Parameters:

userAgent - userAgent or null for none

getVersion

public Map<String,Object> getVersion()

A structure describing which components and their versions of the service are required in order to run the job.

Returns:

value or null for none

setVersion

public Environment setVersion(Map<String,Object> version)

A structure describing which components and their versions of the service are required in order to run the job.

Parameters:

version - version or null for none

getWorkerPools

public List<WorkerPool> getWorkerPools()

The worker pools. At least one "harness" worker pool must be specified in order for the job to have workers.

Returns:

value or null for none

setWorkerPools

public Environment setWorkerPools(List<WorkerPool> workerPools)

The worker pools. At least one "harness" worker pool must be specified in order for the job to have workers.

Parameters:

workerPools - workerPools or null for none

getWorkerRegion

public String getWorkerRegion()

Optional. The Compute Engine region (https://cloud.google.com/compute/docs/regions- zones/regions-zones) in which worker processing should occur, e.g. "us-west1". Mutually exclusive with worker_zone. If neither worker_region nor worker_zone is specified, default to the control plane's region.

Returns:

value or null for none

setWorkerRegion

public Environment setWorkerRegion(String workerRegion)

Optional. The Compute Engine region (https://cloud.google.com/compute/docs/regions- zones/regions-zones) in which worker processing should occur, e.g. "us-west1". Mutually exclusive with worker_zone. If neither worker_region nor worker_zone is specified, default to the control plane's region.

Parameters:

workerRegion - workerRegion or null for none

getWorkerZone

public String getWorkerZone()

Optional. The Compute Engine zone (https://cloud.google.com/compute/docs/regions-zones/regions- zones) in which worker processing should occur, e.g. "us-west1-a". Mutually exclusive with worker_region. If neither worker_region nor worker_zone is specified, a zone in the control plane's region is chosen based on available capacity.

Returns:

value or null for none

setWorkerZone

public Environment setWorkerZone(String workerZone)

Optional. The Compute Engine zone (https://cloud.google.com/compute/docs/regions-zones/regions- zones) in which worker processing should occur, e.g. "us-west1-a". Mutually exclusive with worker_region. If neither worker_region nor worker_zone is specified, a zone in the control plane's region is chosen based on available capacity.

Parameters:

workerZone - workerZone or null for none

set

public Environment set(String fieldName,
                       Object value)

Overrides:

set in class com.google.api.client.json.GenericJson

clone

public Environment clone()

Overrides:

clone in class com.google.api.client.json.GenericJson