Package org.apache.beam.sdk.io.gcp.bigquery

Class RowMutationInformation

java.lang.Object

org.apache.beam.sdk.io.gcp.bigquery.RowMutationInformation

public abstract class RowMutationInformation
extends java.lang.Object

This class indicates how to apply a row update to BigQuery. A sequence number must always be supplied to order the updates. Incorrect sequence numbers will result in unexpected state in the BigQuery table.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	RowMutationInformation.MutationType

Constructor Summary

Constructors

Constructor	Description
RowMutationInformation()

Method Summary

Modifier and Type	Method	Description
abstract java.lang.String	getChangeSequenceNumber()	The value supplied to the BigQuery _CHANGE_SEQUENCE_NUMBER pseudo-column.
abstract RowMutationInformation.MutationType	getMutationType()
abstract @Nullable java.lang.Long	getSequenceNumber()	Deprecated.
static RowMutationInformation	of(RowMutationInformation.MutationType mutationType, long sequenceNumber)	Deprecated.
static RowMutationInformation	of(RowMutationInformation.MutationType mutationType, java.lang.String changeSequenceNumber)	Instantiate RowMutationInformation with RowMutationInformation.MutationType and the {@param changeSequenceNumber}, which sets the BigQuery API _CHANGE_SEQUENCE_NUMBER pseudo column, enabling custom user-supplied ordering of RowMutations.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

RowMutationInformation

public RowMutationInformation()

Method Detail

getMutationType

public abstract RowMutationInformation.MutationType getMutationType()

getSequenceNumber

@Deprecated
public abstract @Nullable java.lang.Long getSequenceNumber()

Deprecated.

The sequence number used to drive the order of applied row mutations. @deprecated getChangeSequenceNumber() replaces this field as the BigQuery API instead supports the use of a string.

getChangeSequenceNumber

public abstract java.lang.String getChangeSequenceNumber()

The value supplied to the BigQuery _CHANGE_SEQUENCE_NUMBER pseudo-column. See of(MutationType, String) for more details.

of

@Deprecated
public static RowMutationInformation of(RowMutationInformation.MutationType mutationType,
                                        long sequenceNumber)

Deprecated.

Instantiate RowMutationInformation with RowMutationInformation.MutationType and the {@param sequenceNumber}. @deprecated - instantiates RowMutationInformation via of(MutationType, String) forwarding the {@param sequenceNumber} value using Long.toHexString(long). {@param sequenceNumber} values < 0 will throw an error.

of

public static RowMutationInformation of(RowMutationInformation.MutationType mutationType,
                                        java.lang.String changeSequenceNumber)

Instantiate RowMutationInformation with RowMutationInformation.MutationType and the {@param changeSequenceNumber}, which sets the BigQuery API _CHANGE_SEQUENCE_NUMBER pseudo column, enabling custom user-supplied ordering of RowMutations.

Requirements for the {@param changeSequenceNumber}:

• fixed format String in hexadecimal format
• do not use hexadecimals encoded from negative numbers
• each hexadecimal string separated into sections by forward slash: /
• up to four sections allowed
• each section is limited to 16 hexadecimal characters: 0-9, A-F, or a-f
• The allowable range supported are values between 0/0/0/0 and FFFFFFFFFFFFFFFF/FFFFFFFFFFFFFFFF/FFFFFFFFFFFFFFFF/FFFFFFFFFFFFFFFF

Below are some {@param changeSequenceNumber} scenarios:

Record #1: {@param changeSequenceNumber}	Record #2: {@param changeSequenceNumber}	BigQuery API compares as
"B"	"ABC"	Record #2 is considered the latest record: 'ABC' > 'B' (i.e. '2748' > '11')
"FFF/B"	"FFF/ABC"	Record #2 is considered the latest record: "FFF/B" > "FFF/ABC" (i.e. "4095/2748" > "4095/11")
"BA/FFFFFFFF"	"ABC"	Record #2 is considered the latest record: "ABC" > "BA/FFFFFFFF" (i.e. "2748" > "186/4294967295")
"FFF/ABC"	"ABC"	Record #1 is considered the latest record: "FFF/ABC" > "ABC" (i.e. "4095/2748" > "2748")
"FFF"	"FFF"	Record #1 and #2 change sequence number identical; BigQuery uses system ingestion time to take precedence over previously ingested records.

Below are some code examples.

• RowMutationInformation.of(UPSERT, "FFF/ABC")
• Using Apache Commons Hex.encodeHexString(byte[]) (Java 17+ users can use HexFormat)RowMutationInformation.of(UPSERT, Hex.encodeHexString("2024-04-30 11:19:44 UTC".getBytes(StandardCharsets.UTF_8)))
• Using Long.toHexString(long): RowMutationInformation.of(DELETE, Long.toHexString(123L))

See https://cloud.google.com/bigquery/docs/change-data-capture#manage_custom_ordering for more details.