Package org.apache.beam.sdk.io.gcp.bigquery

Class BigQueryServicesImpl.DatasetServiceImpl

java.lang.Object

org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl.DatasetServiceImpl

All Implemented Interfaces:

java.lang.AutoCloseable, BigQueryServices.DatasetService

Enclosing class:

BigQueryServicesImpl

public static class BigQueryServicesImpl.DatasetServiceImpl
extends java.lang.Object
implements BigQueryServices.DatasetService

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.beam.sdk.io.gcp.bigquery.BigQueryServices.DatasetService

BigQueryServices.DatasetService.TableMetadataView

Constructor Summary

Constructors

Constructor	Description
DatasetServiceImpl(BigQueryOptions bqOptions)

Method Summary

Modifier and Type	Method	Description
void	close()
void	createDataset(java.lang.String projectId, java.lang.String datasetId, @Nullable java.lang.String location, @Nullable java.lang.String description, @Nullable java.lang.Long defaultTableExpirationMs)	Create a Dataset with the given location, description and default expiration time for tables in the dataset (if null, tables don't expire).
void	createTable(com.google.api.services.bigquery.model.Table table)	Creates the specified table if it does not exist.
void	deleteDataset(java.lang.String projectId, java.lang.String datasetId)	Deletes the dataset specified by the datasetId value.
void	deleteTable(com.google.api.services.bigquery.model.TableReference tableRef)	Deletes the table specified by tableId from the dataset.
com.google.api.services.bigquery.model.Dataset	getDataset(java.lang.String projectId, java.lang.String datasetId)	Gets the specified Dataset resource by dataset ID.
protected static @Nullable com.google.api.client.googleapis.json.GoogleJsonError.ErrorInfo	getErrorInfo(java.io.IOException e)
@Nullable com.google.api.services.bigquery.model.Table	getTable(com.google.api.services.bigquery.model.TableReference tableRef)	Gets the specified Table resource by table ID.
@Nullable com.google.api.services.bigquery.model.Table	getTable(com.google.api.services.bigquery.model.TableReference tableRef, java.util.List<java.lang.String> selectedFields)
@Nullable com.google.api.services.bigquery.model.Table	getTable(com.google.api.services.bigquery.model.TableReference tableRef, java.util.List<java.lang.String> selectedFields, BigQueryServices.DatasetService.TableMetadataView view)
<T> long	insertAll(com.google.api.services.bigquery.model.TableReference ref, java.util.List<org.apache.beam.sdk.values.FailsafeValueInSingleWindow<com.google.api.services.bigquery.model.TableRow,com.google.api.services.bigquery.model.TableRow>> rowList, @Nullable java.util.List<java.lang.String> insertIdList, InsertRetryPolicy retryPolicy, java.util.List<org.apache.beam.sdk.values.ValueInSingleWindow<T>> failedInserts, ErrorContainer<T> errorContainer, boolean skipInvalidRows, boolean ignoreUnknownValues, boolean ignoreInsertIds, java.util.List<org.apache.beam.sdk.values.ValueInSingleWindow<com.google.api.services.bigquery.model.TableRow>> successfulRows)	Inserts TableRows with the specified insertIds if not null.
boolean	isTableEmpty(com.google.api.services.bigquery.model.TableReference tableRef)	Returns true if the table is empty.
com.google.api.services.bigquery.model.Table	patchTableDescription(com.google.api.services.bigquery.model.TableReference tableReference, @Nullable java.lang.String tableDescription)	Patch BigQuery Table description.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DatasetServiceImpl

public DatasetServiceImpl(BigQueryOptions bqOptions)

Method Detail

getTable

public @Nullable com.google.api.services.bigquery.model.Table getTable(com.google.api.services.bigquery.model.TableReference tableRef)
                                                                throws java.io.IOException,
                                                                       java.lang.InterruptedException

Gets the specified Table resource by table ID.

Returns null if the table is not found.

Tries executing the RPC for at most MAX_RPC_RETRIES times until it succeeds.

Specified by:

getTable in interface BigQueryServices.DatasetService

Throws:

java.io.IOException - if it exceeds MAX_RPC_RETRIES attempts.

java.lang.InterruptedException

getTable

public @Nullable com.google.api.services.bigquery.model.Table getTable(com.google.api.services.bigquery.model.TableReference tableRef,
                                                                       java.util.List<java.lang.String> selectedFields)
                                                                throws java.io.IOException,
                                                                       java.lang.InterruptedException

Specified by:

getTable in interface BigQueryServices.DatasetService

Throws:

java.io.IOException

java.lang.InterruptedException

getTable

public @Nullable com.google.api.services.bigquery.model.Table getTable(com.google.api.services.bigquery.model.TableReference tableRef,
                                                                       java.util.List<java.lang.String> selectedFields,
                                                                       BigQueryServices.DatasetService.TableMetadataView view)
                                                                throws java.io.IOException,
                                                                       java.lang.InterruptedException

Specified by:

getTable in interface BigQueryServices.DatasetService

Throws:

java.io.IOException

java.lang.InterruptedException

createTable

public void createTable(com.google.api.services.bigquery.model.Table table)
                 throws java.lang.InterruptedException,
                        java.io.IOException

Creates the specified table if it does not exist.

If a table with the same name already exists in the dataset, the function simply returns. In such a case, the existing table doesn't necessarily have the same schema as specified by the parameter.

Specified by:

createTable in interface BigQueryServices.DatasetService

Throws:

java.io.IOException - if other error than already existing table occurs.

java.lang.InterruptedException

deleteTable

public void deleteTable(com.google.api.services.bigquery.model.TableReference tableRef)
                 throws java.io.IOException,
                        java.lang.InterruptedException

Deletes the table specified by tableId from the dataset. If the table contains data, all the data will be deleted.

Tries executing the RPC for at most MAX_RPC_RETRIES times until it succeeds.

Specified by:

deleteTable in interface BigQueryServices.DatasetService

Throws:

java.io.IOException - if it exceeds MAX_RPC_RETRIES attempts.

java.lang.InterruptedException

isTableEmpty

public boolean isTableEmpty(com.google.api.services.bigquery.model.TableReference tableRef)
                     throws java.io.IOException,
                            java.lang.InterruptedException

Description copied from interface: BigQueryServices.DatasetService

Returns true if the table is empty.

Specified by:

isTableEmpty in interface BigQueryServices.DatasetService

Throws:

java.io.IOException - if the table is not found.

java.lang.InterruptedException

getDataset

public com.google.api.services.bigquery.model.Dataset getDataset(java.lang.String projectId,
                                                                 java.lang.String datasetId)
                                                          throws java.io.IOException,
                                                                 java.lang.InterruptedException

Gets the specified Dataset resource by dataset ID.

Tries executing the RPC for at most MAX_RPC_RETRIES times until it succeeds.

Specified by:

getDataset in interface BigQueryServices.DatasetService

Throws:

java.io.IOException - if it exceeds MAX_RPC_RETRIES attempts.

java.lang.InterruptedException

createDataset

public void createDataset(java.lang.String projectId,
                          java.lang.String datasetId,
                          @Nullable java.lang.String location,
                          @Nullable java.lang.String description,
                          @Nullable java.lang.Long defaultTableExpirationMs)
                   throws java.io.IOException,
                          java.lang.InterruptedException

Create a Dataset with the given location, description and default expiration time for tables in the dataset (if null, tables don't expire).

Tries executing the RPC for at most MAX_RPC_RETRIES times until it succeeds.

Specified by:

createDataset in interface BigQueryServices.DatasetService

Throws:

java.io.IOException - if it exceeds MAX_RPC_RETRIES attempts.

java.lang.InterruptedException

deleteDataset

public void deleteDataset(java.lang.String projectId,
                          java.lang.String datasetId)
                   throws java.io.IOException,
                          java.lang.InterruptedException

Deletes the dataset specified by the datasetId value.

Before you can delete a dataset, you must delete all its tables.

Tries executing the RPC for at most MAX_RPC_RETRIES times until it succeeds.

Specified by:

deleteDataset in interface BigQueryServices.DatasetService

Throws:

java.io.IOException - if it exceeds MAX_RPC_RETRIES attempts.

java.lang.InterruptedException

insertAll

public <T> long insertAll(com.google.api.services.bigquery.model.TableReference ref,
                          java.util.List<org.apache.beam.sdk.values.FailsafeValueInSingleWindow<com.google.api.services.bigquery.model.TableRow,com.google.api.services.bigquery.model.TableRow>> rowList,
                          @Nullable java.util.List<java.lang.String> insertIdList,
                          InsertRetryPolicy retryPolicy,
                          java.util.List<org.apache.beam.sdk.values.ValueInSingleWindow<T>> failedInserts,
                          ErrorContainer<T> errorContainer,
                          boolean skipInvalidRows,
                          boolean ignoreUnknownValues,
                          boolean ignoreInsertIds,
                          java.util.List<org.apache.beam.sdk.values.ValueInSingleWindow<com.google.api.services.bigquery.model.TableRow>> successfulRows)
                   throws java.io.IOException,
                          java.lang.InterruptedException

Description copied from interface: BigQueryServices.DatasetService

Inserts TableRows with the specified insertIds if not null.

If any insert fail permanently according to the retry policy, those rows are added to failedInserts.

Returns the total bytes count of TableRows.

Specified by:

insertAll in interface BigQueryServices.DatasetService

Throws:

java.io.IOException

java.lang.InterruptedException

getErrorInfo

protected static @Nullable com.google.api.client.googleapis.json.GoogleJsonError.ErrorInfo getErrorInfo(java.io.IOException e)

patchTableDescription

public com.google.api.services.bigquery.model.Table patchTableDescription(com.google.api.services.bigquery.model.TableReference tableReference,
                                                                          @Nullable java.lang.String tableDescription)
                                                                   throws java.io.IOException,
                                                                          java.lang.InterruptedException

Description copied from interface: BigQueryServices.DatasetService

Patch BigQuery Table description.

Specified by:

patchTableDescription in interface BigQueryServices.DatasetService

Throws:

java.io.IOException

java.lang.InterruptedException

close

public void close()
           throws java.lang.Exception

Specified by:

close in interface java.lang.AutoCloseable

Throws:

java.lang.Exception