Package org.apache.beam.sdk.io.gcp.datastore

Class DatastoreV1.Write

  • All Implemented Interfaces:
    java.io.Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData
    Enclosing class:
    DatastoreV1
    public static class DatastoreV1.Write
extends org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<com.google.datastore.v1.Entity>,​org.apache.beam.sdk.values.PDone>
    A PTransform that writes Entity objects to Cloud Datastore.
    See Also:
    DatastoreIO, Serialized Form

    • Method Detail

      • withProjectId

        public DatastoreV1.Write withProjectId​(java.lang.String projectId)
        Returns a new DatastoreV1.Write that writes to the Cloud Datastore for the default database.

      • withDatabaseId

        public DatastoreV1.Write withDatabaseId​(java.lang.String databaseId)
        Returns a new DatastoreV1.Write that writes to the Cloud Datastore for the database id.

      • withLocalhost

        public DatastoreV1.Write withLocalhost​(java.lang.String localhost)
        Returns a new DatastoreV1.Write that writes to the Cloud Datastore Emulator running locally on the specified host port.

      • withHintNumWorkers

        public DatastoreV1.Write withHintNumWorkers​(int hintNumWorkers)
        Returns a new DatastoreV1.Write with a different worker count hint for ramp-up throttling. Value is ignored if ramp-up throttling is disabled.

      • withHintNumWorkers

        public DatastoreV1.Write withHintNumWorkers​(org.apache.beam.sdk.options.ValueProvider<java.lang.Integer> hintNumWorkers)
        Same as withHintNumWorkers(int) but with a ValueProvider.

      • withResults

        public DatastoreV1.WriteWithSummary withResults()
        Returns DatastoreV1.WriteWithSummary transform which can be used in Wait.on(PCollection[]) to wait until all data is written.

        Example: write a PCollection to one database and then to another database, making sure that writing a window of data to the second database starts only after the respective window has been fully written to the first database. 

        
 PCollection<Entity> entities = ... ;
 PCollection<DatastoreV1.WriteSuccessSummary> writeSummary =
         entities.apply(DatastoreIO.v1().write().withProjectId(project).withResults());

      • toString

        public java.lang.String toString()
        Overrides:
        toString in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<com.google.datastore.v1.Entity>,​org.apache.beam.sdk.values.PDone>

      • populateDisplayData

        public void populateDisplayData​(org.apache.beam.sdk.transforms.display.DisplayData.Builder builder)
        Specified by:
        populateDisplayData in interface org.apache.beam.sdk.transforms.display.HasDisplayData
        Overrides:
        populateDisplayData in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<com.google.datastore.v1.Entity>,​org.apache.beam.sdk.values.PDone>

      • getProjectId

        public java.lang.String getProjectId()

      • getDatabaseId

        public java.lang.String getDatabaseId()

      • expand

        public org.apache.beam.sdk.values.PDone expand​(org.apache.beam.sdk.values.PCollection<com.google.datastore.v1.Entity> input)
        Specified by:
        expand in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<com.google.datastore.v1.Entity>,​org.apache.beam.sdk.values.PDone>