Package org.apache.beam.sdk.io.gcp.firestore

Class RpcQosOptions

  • All Implemented Interfaces:
    java.io.Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData
    @Immutable
@ThreadSafe
public final class RpcQosOptions
extends java.lang.Object
implements java.io.Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData
    Quality of Service manager options for Firestore RPCs.

    Every RPC which is sent to Cloud Firestore is subject to QoS considerations. Successful, failed, attempted requests are all tracked and directly drive the determination of when to attempt an RPC.

    Configuration of options can be accomplished by passing an instances of RpcQosOptions to the withRpcQosOptions method of each Builder available in FirestoreV1.

    A new instance of RpcQosOptions.Builder can be created via newBuilder(). A default instance of RpcQosOptions can be created via defaultOptions().

    See Also:
    FirestoreV1, FirestoreV1.Transform.Builder.withRpcQosOptions(RpcQosOptions), FirestoreV1.Transform.Builder.withRpcQosOptions(RpcQosOptions), FirestoreV1.Transform.Builder.withRpcQosOptions(RpcQosOptions), FirestoreV1.Transform.Builder.withRpcQosOptions(RpcQosOptions), FirestoreV1.Transform.Builder.withRpcQosOptions(RpcQosOptions), FirestoreV1.Transform.Builder.withRpcQosOptions(RpcQosOptions), Standard limits, Designing for scale, Serialized Form

    • Method Detail

      • populateDisplayData

        public void populateDisplayData​(org.apache.beam.sdk.transforms.display.DisplayData.Builder builder)
        Specified by:
        populateDisplayData in interface org.apache.beam.sdk.transforms.display.HasDisplayData

      • getMaxAttempts

        public int getMaxAttempts()
        The maximum number of times a request will be attempted for a complete successful result.

        For a stream based response, the full stream read must complete within the specified number of attempts. Restarting a stream will count as a new attempt.

        Default Value: 5

        See Also:
        RpcQosOptions.Builder.withMaxAttempts(int)

      • getBatchInitialCount

        public int getBatchInitialCount()
        The initial size of a batch; used in the absence of the QoS system having significant data to determine a better batch size.

        Default Value: 20

        See Also:
        RpcQosOptions.Builder.withBatchInitialCount(int)

      • getBatchMaxCount

        public int getBatchMaxCount()
        The maximum number of writes to include in a batch. (Used in the absence of the QoS system having significant data to determine a better value). The actual number of writes per request may be lower if we reach batchMaxBytes.

        Default Value: 500

        See Also:
        RpcQosOptions.Builder.withBatchMaxCount(int)

      • getBatchMaxBytes

        public long getBatchMaxBytes()
        The maximum number of bytes to include in a batch. (Used in the absence of the QoS system having significant data to determine a better value). The actual number of bytes per request may be lower if batchMaxCount is reached first.

        Default Value: 9.5 MiB

        See Also:
        RpcQosOptions.Builder.withBatchMaxBytes(long)

      • getBatchTargetLatency

        public org.joda.time.Duration getBatchTargetLatency()
        Target latency for batch requests. It aims for a target response time per RPC: the response times for previous RPCs and the number of writes contained in them, calculates a rolling average time-per-write, and chooses the number of writes for future requests to hit the target time.

        Default Value: 5 sec

        See Also:
        RpcQosOptions.Builder.withBatchTargetLatency(Duration)

      • getHintMaxNumWorkers

        public int getHintMaxNumWorkers()
        A hint to the QoS system for the intended max number of workers for a pipeline. The provided value can be used to try to scale calculations to values appropriate for the pipeline as a whole.

        If you are running your pipeline on Cloud Dataflow this parameter should be set to the same value as maxNumWorkers

        Default Value: 500

        See Also:
        RpcQosOptions.Builder.withBatchMaxCount(int)

      • isShouldReportDiagnosticMetrics

        public boolean isShouldReportDiagnosticMetrics()
        Whether additional diagnostic metrics should be reported for a Transform.

        If true, additional detailed diagnostic metrics will be reported for the RPC QoS subsystem.

        This parameter should be used with care as it will output ~40 additional custom metrics which will count toward any possible pipeline metrics limits (For example Dataflow Custom Metrics limits and Cloud Monitoring custom metrics quota)

        Default Value: false

        See Also:
        RpcQosOptions.Builder.withReportDiagnosticMetrics()

      • equals

        public boolean equals​(@Nullable java.lang.Object o)
        Overrides:
        equals in class java.lang.Object

      • hashCode

        public int hashCode()
        Overrides:
        hashCode in class java.lang.Object

      • toString

        public java.lang.String toString()
        Overrides:
        toString in class java.lang.Object