Package org.apache.beam.sdk.io.gcp.bigquery

Class BigQueryIO.Read

  • All Implemented Interfaces:
    java.io.Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData
    Enclosing class:
    BigQueryIO
    public static class BigQueryIO.Read
extends org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PBegin,​org.apache.beam.sdk.values.PCollection<com.google.api.services.bigquery.model.TableRow>>
    Implementation of BigQueryIO.read().
    See Also:
    Serialized Form

    • Field Summary

      • Fields inherited from class org.apache.beam.sdk.transforms.PTransform

        annotations, displayData, name, resourceHints

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      org.apache.beam.sdk.values.PCollection<com.google.api.services.bigquery.model.TableRow> expand​(org.apache.beam.sdk.values.PBegin input)  
      BigQueryIO.Read from​(com.google.api.services.bigquery.model.TableReference table)
      Read from table specified by a TableReference.
      BigQueryIO.Read from​(java.lang.String tableSpec)
      Reads a BigQuery table specified as "[project_id]:[dataset_id].[table_id]" or "[dataset_id].[table_id]" for tables within the current project.
      BigQueryIO.Read from​(org.apache.beam.sdk.options.ValueProvider<java.lang.String> tableSpec)
      Same as from(String), but with a ValueProvider.
      BigQueryIO.Read fromQuery​(java.lang.String query)
      Reads results received after executing the given query.
      BigQueryIO.Read fromQuery​(org.apache.beam.sdk.options.ValueProvider<java.lang.String> query)
      Same as fromQuery(String), but with a ValueProvider.
      @Nullable com.google.api.services.bigquery.model.TableReference getTable()
      Returns the table to read, or null if reading from a query instead.
      @Nullable org.apache.beam.sdk.options.ValueProvider<com.google.api.services.bigquery.model.TableReference> getTableProvider()
      Returns the table to read, or null if reading from a query instead.
      void populateDisplayData​(org.apache.beam.sdk.transforms.display.DisplayData.Builder builder)  
      BigQueryIO.Read usingStandardSql()
      Enables BigQuery's Standard SQL dialect when reading from a query.
      BigQueryIO.Read withoutResultFlattening()
      Disable flattening of query results.
      BigQueryIO.Read withoutValidation()
      Disable validation that the table exists or the query succeeds prior to pipeline submission.
      BigQueryIO.Read withTemplateCompatibility()
      Use new template-compatible source implementation.
      BigQueryIO.Read withTestServices​(BigQueryServices testServices)  

      • Methods inherited from class org.apache.beam.sdk.transforms.PTransform

        addAnnotation, compose, compose, getAdditionalInputs, getAnnotations, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, setDisplayData, setResourceHints, toString, validate, validate

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

    • Method Detail

      • expand

        public org.apache.beam.sdk.values.PCollection<com.google.api.services.bigquery.model.TableRow> expand​(org.apache.beam.sdk.values.PBegin input)
        Specified by:
        expand in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PBegin,​org.apache.beam.sdk.values.PCollection<com.google.api.services.bigquery.model.TableRow>>

      • populateDisplayData

        public void populateDisplayData​(org.apache.beam.sdk.transforms.display.DisplayData.Builder builder)
        Specified by:
        populateDisplayData in interface org.apache.beam.sdk.transforms.display.HasDisplayData
        Overrides:
        populateDisplayData in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PBegin,​org.apache.beam.sdk.values.PCollection<com.google.api.services.bigquery.model.TableRow>>

      • getTableProvider

        public @Nullable org.apache.beam.sdk.options.ValueProvider<com.google.api.services.bigquery.model.TableReference> getTableProvider()
        Returns the table to read, or null if reading from a query instead.

      • getTable

        public @Nullable com.google.api.services.bigquery.model.TableReference getTable()
        Returns the table to read, or null if reading from a query instead.

      • from

        public BigQueryIO.Read from​(java.lang.String tableSpec)
        Reads a BigQuery table specified as "[project_id]:[dataset_id].[table_id]" or "[dataset_id].[table_id]" for tables within the current project.

      • from

        public BigQueryIO.Read from​(org.apache.beam.sdk.options.ValueProvider<java.lang.String> tableSpec)
        Same as from(String), but with a ValueProvider.

      • from

        public BigQueryIO.Read from​(com.google.api.services.bigquery.model.TableReference table)
        Read from table specified by a TableReference.

      • fromQuery

        public BigQueryIO.Read fromQuery​(java.lang.String query)
        Reads results received after executing the given query.

        By default, the query results will be flattened -- see "flattenResults" in the Jobs documentation for more information. To disable flattening, use withoutResultFlattening().

        By default, the query will use BigQuery's legacy SQL dialect. To use the BigQuery Standard SQL dialect, use usingStandardSql().

      • fromQuery

        public BigQueryIO.Read fromQuery​(org.apache.beam.sdk.options.ValueProvider<java.lang.String> query)
        Same as fromQuery(String), but with a ValueProvider.

      • withoutValidation

        public BigQueryIO.Read withoutValidation()
        Disable validation that the table exists or the query succeeds prior to pipeline submission. Basic validation (such as ensuring that a query or table is specified) still occurs.

      • usingStandardSql

        public BigQueryIO.Read usingStandardSql()
        Enables BigQuery's Standard SQL dialect when reading from a query.

        Only valid when a query is used (fromQuery(java.lang.String)). Setting this option when reading from a table will cause an error during validation.

      • withTemplateCompatibility

        public BigQueryIO.Read withTemplateCompatibility()
        Use new template-compatible source implementation.

        Use new template-compatible source implementation. This implementation is compatible with repeated template invocations. It does not support dynamic work rebalancing.