Package org.apache.beam.sdk.io.gcp.testing

Class BigqueryClient

  • @Internal
public class BigqueryClient
extends java.lang.Object
    A wrapper class to call Bigquery API calls.

    Example:

    Get a new Bigquery client: 

    [
    BigqueryClient client = BigqueryClient.getNewBigqueryClient(applicationName);
 ]

    Execute a query with retries: 

    [
    QueryResponse response = client.queryWithRetries(queryString, projectId);
 ]

    Create a new dataset in one project: 

    [
    client.createNewDataset(projectId, datasetId);
 ]

    Delete a dataset in one project, included its all tables: 

    [
    client.deleteDataset(projectId, datasetId);
 ]

    Create a new table 

    [
    client.createNewTable(projectId, datasetId, newTable)
 ]

    Insert data into table 

    [
    client.insertDataToTable(projectId, datasetId, tableName, rows)
 ]

    • Constructor Summary

      Constructors 
      Constructor Description
      BigqueryClient​(java.lang.String applicationName)  

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void createNewDataset​(java.lang.String projectId, java.lang.String datasetId)
      Creates a new dataset.
      void createNewDataset​(java.lang.String projectId, java.lang.String datasetId, @Nullable java.lang.Long defaultTableExpirationMs)
      Creates a new dataset with defaultTableExpirationMs.
      void createNewDataset​(java.lang.String projectId, java.lang.String datasetId, @Nullable java.lang.Long defaultTableExpirationMs, @Nullable java.lang.String location)
      Creates a new dataset with defaultTableExpirationMs and in a specified location (GCP region).
      void createNewTable​(java.lang.String projectId, java.lang.String datasetId, com.google.api.services.bigquery.model.Table newTable)  
      void deleteDataset​(java.lang.String projectId, java.lang.String datasetId)  
      void deleteTable​(java.lang.String projectId, java.lang.String datasetId, java.lang.String tableName)  
      static BigqueryClient getClient​(java.lang.String applicationName)  
      static com.google.api.services.bigquery.Bigquery getNewBigqueryClient​(java.lang.String applicationName)  
      com.google.api.services.bigquery.model.Table getTableResource​(java.lang.String projectId, java.lang.String datasetId, java.lang.String tableId)  
      void insertDataToTable​(java.lang.String projectId, java.lang.String datasetId, java.lang.String tableName, java.util.List<java.util.Map<java.lang.String,​java.lang.Object>> rows)
      Inserts rows to a table using a BigQuery streaming write.
      java.util.List<com.google.api.services.bigquery.model.TableRow> queryUnflattened​(java.lang.String query, java.lang.String projectId, boolean typed, boolean useStandardSql)
      Performs a query without flattening results.
      java.util.List<com.google.api.services.bigquery.model.TableRow> queryUnflattened​(java.lang.String query, java.lang.String projectId, boolean typed, boolean useStandardSql, @Nullable java.lang.String location)
      Performs a query without flattening results.
      com.google.api.services.bigquery.model.QueryResponse queryWithRetries​(java.lang.String query, java.lang.String projectId)  
      com.google.api.services.bigquery.model.QueryResponse queryWithRetries​(java.lang.String query, java.lang.String projectId, boolean typed)  
      com.google.api.services.bigquery.model.QueryResponse queryWithRetriesUsingStandardSql​(java.lang.String query, java.lang.String projectId)  
      void updateTableSchema​(java.lang.String projectId, java.lang.String datasetId, java.lang.String tableId, com.google.api.services.bigquery.model.TableSchema newSchema)  

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • BigqueryClient

        public BigqueryClient​(java.lang.String applicationName)

    • Method Detail

      • getNewBigqueryClient

        public static com.google.api.services.bigquery.Bigquery getNewBigqueryClient​(java.lang.String applicationName)

      • getClient

        public static BigqueryClient getClient​(java.lang.String applicationName)

      • queryWithRetries

        @Nonnull
public com.google.api.services.bigquery.model.QueryResponse queryWithRetries​(java.lang.String query,
                                                                             java.lang.String projectId)
                                                                      throws java.io.IOException,
                                                                             java.lang.InterruptedException
        Throws:
        java.io.IOException
        java.lang.InterruptedException

      • queryWithRetriesUsingStandardSql

        @Nonnull
public com.google.api.services.bigquery.model.QueryResponse queryWithRetriesUsingStandardSql​(java.lang.String query,
                                                                                             java.lang.String projectId)
                                                                                      throws java.io.IOException,
                                                                                             java.lang.InterruptedException
        Throws:
        java.io.IOException
        java.lang.InterruptedException

      • queryUnflattened

        @Nonnull
public java.util.List<com.google.api.services.bigquery.model.TableRow> queryUnflattened​(java.lang.String query,
                                                                                        java.lang.String projectId,
                                                                                        boolean typed,
                                                                                        boolean useStandardSql)
                                                                                 throws java.io.IOException,
                                                                                        java.lang.InterruptedException
        Performs a query without flattening results.
        Throws:
        java.io.IOException
        java.lang.InterruptedException

      • queryUnflattened

        @Nonnull
public java.util.List<com.google.api.services.bigquery.model.TableRow> queryUnflattened​(java.lang.String query,
                                                                                        java.lang.String projectId,
                                                                                        boolean typed,
                                                                                        boolean useStandardSql,
                                                                                        @Nullable java.lang.String location)
                                                                                 throws java.io.IOException,
                                                                                        java.lang.InterruptedException
        Performs a query without flattening results. May choose a location (GCP region) to perform this operation in.
        Throws:
        java.io.IOException
        java.lang.InterruptedException

      • queryWithRetries

        @Nonnull
public com.google.api.services.bigquery.model.QueryResponse queryWithRetries​(java.lang.String query,
                                                                             java.lang.String projectId,
                                                                             boolean typed)
                                                                      throws java.io.IOException,
                                                                             java.lang.InterruptedException
        Throws:
        java.io.IOException
        java.lang.InterruptedException

      • createNewDataset

        public void createNewDataset​(java.lang.String projectId,
                             java.lang.String datasetId)
                      throws java.io.IOException,
                             java.lang.InterruptedException
        Creates a new dataset.
        Throws:
        java.io.IOException
        java.lang.InterruptedException

      • createNewDataset

        public void createNewDataset​(java.lang.String projectId,
                             java.lang.String datasetId,
                             @Nullable java.lang.Long defaultTableExpirationMs)
                      throws java.io.IOException,
                             java.lang.InterruptedException
        Creates a new dataset with defaultTableExpirationMs.
        Throws:
        java.io.IOException
        java.lang.InterruptedException

      • createNewDataset

        public void createNewDataset​(java.lang.String projectId,
                             java.lang.String datasetId,
                             @Nullable java.lang.Long defaultTableExpirationMs,
                             @Nullable java.lang.String location)
                      throws java.io.IOException,
                             java.lang.InterruptedException
        Creates a new dataset with defaultTableExpirationMs and in a specified location (GCP region).
        Throws:
        java.io.IOException
        java.lang.InterruptedException

      • deleteTable

        public void deleteTable​(java.lang.String projectId,
                        java.lang.String datasetId,
                        java.lang.String tableName)

      • deleteDataset

        public void deleteDataset​(java.lang.String projectId,
                          java.lang.String datasetId)

      • createNewTable

        public void createNewTable​(java.lang.String projectId,
                           java.lang.String datasetId,
                           com.google.api.services.bigquery.model.Table newTable)
                    throws java.io.IOException,
                           java.lang.InterruptedException
        Throws:
        java.io.IOException
        java.lang.InterruptedException

      • insertDataToTable

        public void insertDataToTable​(java.lang.String projectId,
                              java.lang.String datasetId,
                              java.lang.String tableName,
                              java.util.List<java.util.Map<java.lang.String,​java.lang.Object>> rows)
                       throws java.io.IOException,
                              java.lang.InterruptedException
        Inserts rows to a table using a BigQuery streaming write.
        Throws:
        java.io.IOException
        java.lang.InterruptedException

      • getTableResource

        public com.google.api.services.bigquery.model.Table getTableResource​(java.lang.String projectId,
                                                                     java.lang.String datasetId,
                                                                     java.lang.String tableId)
                                                              throws java.io.IOException,
                                                                     java.lang.InterruptedException
        Throws:
        java.io.IOException
        java.lang.InterruptedException

      • updateTableSchema

        public void updateTableSchema​(java.lang.String projectId,
                              java.lang.String datasetId,
                              java.lang.String tableId,
                              com.google.api.services.bigquery.model.TableSchema newSchema)
                       throws java.io.IOException
        Throws:
        java.io.IOException