Package org.apache.beam.sdk.io.gcp.bigtable

Class BigtableIO.Write

  • All Implemented Interfaces:
    java.io.Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData
    Enclosing class:
    BigtableIO
    public abstract static class BigtableIO.Write
extends org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<com.google.protobuf.ByteString,​java.lang.Iterable<com.google.bigtable.v2.Mutation>>>,​org.apache.beam.sdk.values.PDone>
    A PTransform that writes to Google Cloud Bigtable. See the class-level Javadoc on BigtableIO for more information.
    See Also:
    BigtableIO, Serialized Form

    • Constructor Detail

      • Write

        public Write()

    • Method Detail

      • getBigtableOptions

        @Deprecated
public @Nullable com.google.cloud.bigtable.config.BigtableOptions getBigtableOptions()
        Deprecated.
        write options are configured directly on BigtableIO.write(). Use populateDisplayData(DisplayData.Builder) to view the current configurations.
        Returns the Google Cloud Bigtable instance being written to, and other parameters.

      • withTableId

        public BigtableIO.Write withTableId​(org.apache.beam.sdk.options.ValueProvider<java.lang.String> tableId)
        Returns a new BigtableIO.Write that will write to the specified table.

        Does not modify this object.

      • withTableId

        public BigtableIO.Write withTableId​(java.lang.String tableId)
        Returns a new BigtableIO.Write that will write to the specified table.

        Does not modify this object.

      • withAppProfileId

        public BigtableIO.Write withAppProfileId​(org.apache.beam.sdk.options.ValueProvider<java.lang.String> appProfileId)
        Returns a new BigtableIO.Write that will write using the specified app profile id.

        Remember that in order to use single-row transactions, this must use a single-cluster routing policy.

        Does not modify this object.

      • withAppProfileId

        public BigtableIO.Write withAppProfileId​(java.lang.String appProfileId)
        Returns a new BigtableIO.Write that will write using the specified app profile id.

        Remember that in order to use single-row transactions, this must use a single-cluster routing policy.

        Does not modify this object.

      • withoutValidation

        public BigtableIO.Write withoutValidation()
        Disables validation that the table being written to exists.

      • withEmulator

        public BigtableIO.Write withEmulator​(java.lang.String emulatorHost)
        Returns a new BigtableIO.Write that will use an official Bigtable emulator.

        This is used for testing.

      • withAttemptTimeout

        public BigtableIO.Write withAttemptTimeout​(org.joda.time.Duration timeout)
        Returns a new BigtableIO.Write with the attempt timeout. Attempt timeout controls the timeout for each remote call.

        Does not modify this object.

      • withOperationTimeout

        public BigtableIO.Write withOperationTimeout​(org.joda.time.Duration timeout)
        Returns a new BigtableIO.Write with the operation timeout. Operation timeout has ultimate control over how long the logic should keep trying the remote call until it gives up completely.

        Does not modify this object.

      • withMaxElementsPerBatch

        public BigtableIO.Write withMaxElementsPerBatch​(long size)
        Returns a new BigtableIO.Write with the max elements a batch can have. After this many elements are accumulated, they will be wrapped up in a batch and sent to Bigtable.

        Does not modify this object.

      • withMaxBytesPerBatch

        public BigtableIO.Write withMaxBytesPerBatch​(long size)
        Returns a new BigtableIO.Write with the max bytes a batch can have. After this many bytes are accumulated, the elements will be wrapped up in a batch and sent to Bigtable.

        Does not modify this object.

      • withMaxOutstandingElements

        public BigtableIO.Write withMaxOutstandingElements​(long count)
        Returns a new BigtableIO.Write with the max number of outstanding elements allowed before enforcing flow control.

        Does not modify this object.

      • withMaxOutstandingBytes

        public BigtableIO.Write withMaxOutstandingBytes​(long bytes)
        Returns a new BigtableIO.Write with the max number of outstanding bytes allowed before enforcing flow control.

        Does not modify this object.

      • withFlowControl

        public BigtableIO.Write withFlowControl​(boolean enableFlowControl)
        Returns a new BigtableIO.Write with flow control enabled if enableFlowControl is true.

        When enabled, traffic to Bigtable is automatically rate-limited to prevent overloading Bigtable clusters while keeping enough load to trigger Bigtable Autoscaling (if enabled) to provision more nodes as needed. It is different from the flow control set by withMaxOutstandingElements(long) and withMaxOutstandingBytes(long), which is always enabled on batch writes and limits the number of outstanding requests to the Bigtable server.

        Does not modify this object.

      • withThrottlingTargetMs

        @Deprecated
public BigtableIO.Write withThrottlingTargetMs​(int throttlingTargetMs)
        Deprecated.
        This method has been deprecated in Beam 2.60.0. It does not have an effect.

      • withThrottlingReportTargetMs

        @Deprecated
public BigtableIO.Write withThrottlingReportTargetMs​(int throttlingReportTargetMs)
        Deprecated.
        This method has been deprecated in Beam 2.60.0. It does not have an effect.

      • withErrorHandler

        public BigtableIO.Write withErrorHandler​(org.apache.beam.sdk.transforms.errorhandling.ErrorHandler<org.apache.beam.sdk.transforms.errorhandling.BadRecord,​?> badRecordErrorHandler)

      • expand

        public org.apache.beam.sdk.values.PDone expand​(org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<com.google.protobuf.ByteString,​java.lang.Iterable<com.google.bigtable.v2.Mutation>>> input)
        Specified by:
        expand in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<com.google.protobuf.ByteString,​java.lang.Iterable<com.google.bigtable.v2.Mutation>>>,​org.apache.beam.sdk.values.PDone>

      • validate

        public void validate​(org.apache.beam.sdk.options.PipelineOptions options)
        Overrides:
        validate in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<com.google.protobuf.ByteString,​java.lang.Iterable<com.google.bigtable.v2.Mutation>>>,​org.apache.beam.sdk.values.PDone>

      • populateDisplayData

        public void populateDisplayData​(org.apache.beam.sdk.transforms.display.DisplayData.Builder builder)
        Specified by:
        populateDisplayData in interface org.apache.beam.sdk.transforms.display.HasDisplayData
        Overrides:
        populateDisplayData in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<com.google.protobuf.ByteString,​java.lang.Iterable<com.google.bigtable.v2.Mutation>>>,​org.apache.beam.sdk.values.PDone>

      • toString

        public final java.lang.String toString()
        Overrides:
        toString in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<com.google.protobuf.ByteString,​java.lang.Iterable<com.google.bigtable.v2.Mutation>>>,​org.apache.beam.sdk.values.PDone>