Package org.apache.beam.sdk.io.gcp.pubsub

Class PubsubReadSchemaTransformConfiguration

  • @DefaultSchema(org.apache.beam.sdk.schemas.AutoValueSchema.class)
public abstract class PubsubReadSchemaTransformConfiguration
extends java.lang.Object
    Configuration for reading from Pub/Sub.

    Internal only: This class is actively being worked on, and it will likely change. We provide no backwards compatibility guarantees, and it should not be implemented outside the Beam repository.

    • Constructor Detail

      • PubsubReadSchemaTransformConfiguration

        public PubsubReadSchemaTransformConfiguration()

    • Method Detail

      • getTopic

        @SchemaFieldDescription("The name of the topic to consume data from. If a topic is specified,  will create a new subscription for that topic and start consuming from that point. Either a topic or a subscription must be provided. Format: projects/${PROJECT}/topics/${TOPIC}")
@Nullable
public abstract java.lang.String getTopic()

      • getSubscription

        @SchemaFieldDescription("The name of the subscription to consume data. Either a topic or subscription must be provided. Format: projects/${PROJECT}/subscriptions/${SUBSCRIPTION}")
@Nullable
public abstract java.lang.String getSubscription()

      • getFormat

        @SchemaFieldDescription("The encoding format for the data stored in Pubsub. Valid options are: RAW,AVRO,JSON")
public abstract java.lang.String getFormat()

      • getSchema

        @SchemaFieldDescription("The schema in which the data is encoded in the Pubsub topic. For AVRO data, this is a schema defined with AVRO schema syntax (https://avro.apache.org/docs/1.10.2/spec.html#schemas). For JSON data, this is a schema defined with JSON-schema syntax (https://json-schema.org/).")
public abstract java.lang.String getSchema()

      • getAttributes

        @SchemaFieldDescription("Any additional pubsub attributes that should be populated as String fields in the ouptut rows.")
@Nullable
public abstract java.util.List<java.lang.String> getAttributes()

      • getAttributesMap

        @SchemaFieldDescription("Any additional field that should be populated with the full set of PubSub attributes.")
@Nullable
public abstract java.lang.String getAttributesMap()

      • getIdAttribute

        @SchemaFieldDescription("When reading from Cloud Pub/Sub where unique record identifiers are provided as Pub/Sub message attributes, specifies the name of the attribute containing the unique identifier. The value of the attribute can be any string that uniquely identifies this record. Pub/Sub cannot guarantee that no duplicate data will be delivered on the Pub/Sub stream. If idAttribute is not provided, Beam cannot guarantee that no duplicate data will be delivered, and deduplication of the stream will be strictly best effort.")
@Nullable
public abstract java.lang.String getIdAttribute()

      • getTimestampAttribute

        @SchemaFieldDescription("Specifies the name of the attribute that contains the timestamp, if any. The timestamp value is expected to be represented in the attribute as either (1) a numerical value representing the number of milliseconds since the Unix epoch. For example, if using the Joda time classes, Instant.getMillis() returns the correct value for this attribute. or (2) a String in RFC 3339 format. For example, 2015-10-29T23:41:41.123Z. The sub-second component of the timestamp is optional, and digits beyond the first three (i.e., time units smaller than milliseconds) will be ignored.")
@Nullable
public abstract java.lang.String getTimestampAttribute()

      • getClock

        @Nullable
public abstract com.google.api.client.util.Clock getClock()