software.amazon.kinesis.leases.dynamodb

Class DynamoDBLeaseRefresher

java.lang.Object

software.amazon.kinesis.leases.dynamodb.DynamoDBLeaseRefresher

All Implemented Interfaces:

LeaseRefresher

public class DynamoDBLeaseRefresher
extends Object
implements LeaseRefresher

An implementation of LeaseRefresher that uses DynamoDB.

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	consistentReads
protected software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient	dynamoDBClient
protected LeaseSerializer	serializer
protected String	table

Constructor Summary

Constructors

Constructor and Description
DynamoDBLeaseRefresher(String table, software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient dynamoDBClient, LeaseSerializer serializer, boolean consistentReads) Deprecated.
DynamoDBLeaseRefresher(String table, software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient dynamoDBClient, LeaseSerializer serializer, boolean consistentReads, @NonNull TableCreatorCallback tableCreatorCallback) Deprecated.
DynamoDBLeaseRefresher(String table, software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient dynamoDBClient, LeaseSerializer serializer, boolean consistentReads, @NonNull TableCreatorCallback tableCreatorCallback, Duration dynamoDbRequestTimeout) Deprecated.
DynamoDBLeaseRefresher(String table, software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient dynamoDBClient, LeaseSerializer serializer, boolean consistentReads, @NonNull TableCreatorCallback tableCreatorCallback, Duration dynamoDbRequestTimeout, software.amazon.awssdk.services.dynamodb.model.BillingMode billingMode, boolean leaseTableDeletionProtectionEnabled) Deprecated.
DynamoDBLeaseRefresher(String table, software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient dynamoDBClient, LeaseSerializer serializer, boolean consistentReads, @NonNull TableCreatorCallback tableCreatorCallback, Duration dynamoDbRequestTimeout, software.amazon.awssdk.services.dynamodb.model.BillingMode billingMode, boolean leaseTableDeletionProtectionEnabled, Collection<software.amazon.awssdk.services.dynamodb.model.Tag> tags) Constructor.

Method Summary

Modifier and Type	Method and Description
protected DependencyException	convertAndRethrowExceptions(String operation, String leaseKey, Exception e)
boolean	createLeaseIfNotExists(@NonNull Lease lease) Create a new lease.
boolean	createLeaseTableIfNotExists() Creates the table that will store leases.
boolean	createLeaseTableIfNotExists(@NonNull Long readCapacity, @NonNull Long writeCapacity) Creates the table that will store leases.
void	deleteAll() Delete all leases from DynamoDB.
void	deleteLease(@NonNull Lease lease) Delete the given lease from DynamoDB.
boolean	evictLease(@NonNull Lease lease) Evict the current owner of lease by setting owner to null.
ExtendedSequenceNumber	getCheckpoint(String leaseKey) Gets the current checkpoint of the shard.
Lease	getLease(@NonNull String leaseKey)
boolean	isLeaseTableEmpty() Check (synchronously) if there are any leases in the lease table.
boolean	leaseTableExists()
List<Lease>	listLeases() List all objects in table synchronously.
List<Lease>	listLeasesForStream(StreamIdentifier streamIdentifier) List all leases for a given stream synchronously.
boolean	renewLease(@NonNull Lease lease) Renew a lease by incrementing the lease counter.
boolean	takeLease(@NonNull Lease lease, @NonNull String owner) Take a lease for the given owner by incrementing its leaseCounter and setting its owner field.
boolean	updateLease(@NonNull Lease lease) Update application-specific fields of the given lease in DynamoDB.
void	updateLeaseWithMetaInfo(Lease lease, UpdateField updateField) Update application-specific fields of the given lease in DynamoDB.
boolean	waitUntilLeaseTableExists(long secondsBetweenPolls, long timeoutSeconds) Blocks until the lease table exists by polling leaseTableExists.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

table

protected final String table

dynamoDBClient

protected final software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient dynamoDBClient

serializer

protected final LeaseSerializer serializer

consistentReads

protected final boolean consistentReads

Constructor Detail

DynamoDBLeaseRefresher

@Deprecated
public DynamoDBLeaseRefresher(String table,
                                          software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient dynamoDBClient,
                                          LeaseSerializer serializer,
                                          boolean consistentReads)

Deprecated.

Constructor.

NOTE: This constructor is deprecated and will be removed in a future release.

Parameters:

table -

dynamoDBClient -

serializer -

consistentReads -

DynamoDBLeaseRefresher

@Deprecated
public DynamoDBLeaseRefresher(String table,
                                          software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient dynamoDBClient,
                                          LeaseSerializer serializer,
                                          boolean consistentReads,
                                          @NonNull
                                          @NonNull TableCreatorCallback tableCreatorCallback)

Deprecated.

Constructor.

Parameters:

table -

dynamoDBClient -

serializer -

consistentReads -

tableCreatorCallback -

DynamoDBLeaseRefresher

@Deprecated
public DynamoDBLeaseRefresher(String table,
                                          software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient dynamoDBClient,
                                          LeaseSerializer serializer,
                                          boolean consistentReads,
                                          @NonNull
                                          @NonNull TableCreatorCallback tableCreatorCallback,
                                          Duration dynamoDbRequestTimeout)

Deprecated.

Constructor.

Parameters:

table -

dynamoDBClient -

serializer -

consistentReads -

tableCreatorCallback -

dynamoDbRequestTimeout -

DynamoDBLeaseRefresher

@Deprecated
public DynamoDBLeaseRefresher(String table,
                                          software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient dynamoDBClient,
                                          LeaseSerializer serializer,
                                          boolean consistentReads,
                                          @NonNull
                                          @NonNull TableCreatorCallback tableCreatorCallback,
                                          Duration dynamoDbRequestTimeout,
                                          software.amazon.awssdk.services.dynamodb.model.BillingMode billingMode,
                                          boolean leaseTableDeletionProtectionEnabled)

Deprecated.

Constructor.

Parameters:

table -

dynamoDBClient -

serializer -

consistentReads -

tableCreatorCallback -

dynamoDbRequestTimeout -

billingMode -

leaseTableDeletionProtectionEnabled -

DynamoDBLeaseRefresher

public DynamoDBLeaseRefresher(String table,
                              software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient dynamoDBClient,
                              LeaseSerializer serializer,
                              boolean consistentReads,
                              @NonNull
                              @NonNull TableCreatorCallback tableCreatorCallback,
                              Duration dynamoDbRequestTimeout,
                              software.amazon.awssdk.services.dynamodb.model.BillingMode billingMode,
                              boolean leaseTableDeletionProtectionEnabled,
                              Collection<software.amazon.awssdk.services.dynamodb.model.Tag> tags)

Constructor.

Parameters:

table -

dynamoDBClient -

serializer -

consistentReads -

tableCreatorCallback -

dynamoDbRequestTimeout -

billingMode -

leaseTableDeletionProtectionEnabled -

tags -

Method Detail

createLeaseTableIfNotExists

public boolean createLeaseTableIfNotExists(@NonNull
                                           @NonNull Long readCapacity,
                                           @NonNull
                                           @NonNull Long writeCapacity)
                                    throws ProvisionedThroughputException,
                                           DependencyException

Creates the table that will store leases. Succeeds if table already exists. Deprecated. Use LeaseRefresher.createLeaseTableIfNotExists().

Specified by:

createLeaseTableIfNotExists in interface LeaseRefresher

Returns:

true if we created a new table (table didn't exist before)

Throws:

ProvisionedThroughputException - if we cannot create the lease table due to per-AWS-account capacity restrictions.

DependencyException - if DynamoDB createTable fails in an unexpected way

createLeaseTableIfNotExists

public boolean createLeaseTableIfNotExists()
                                    throws ProvisionedThroughputException,
                                           DependencyException

Creates the table that will store leases. Table is now created in PayPerRequest billing mode by default. Succeeds if table already exists.

Specified by:

createLeaseTableIfNotExists in interface LeaseRefresher

Returns:

true if we created a new table (table didn't exist before)

Throws:

ProvisionedThroughputException - if we cannot create the lease table due to per-AWS-account capacity restrictions.

DependencyException - if DynamoDB createTable fails in an unexpected way

leaseTableExists

public boolean leaseTableExists()
                         throws DependencyException

Specified by:

leaseTableExists in interface LeaseRefresher

Returns:

true if the lease table already exists.

Throws:

DependencyException - if DynamoDB describeTable fails in an unexpected way

waitUntilLeaseTableExists

public boolean waitUntilLeaseTableExists(long secondsBetweenPolls,
                                         long timeoutSeconds)
                                  throws DependencyException

Description copied from interface: LeaseRefresher

Blocks until the lease table exists by polling leaseTableExists.

Specified by:

waitUntilLeaseTableExists in interface LeaseRefresher

Parameters:

secondsBetweenPolls - time to wait between polls in seconds

timeoutSeconds - total time to wait in seconds

Returns:

true if table exists, false if timeout was reached

Throws:

DependencyException - if DynamoDB describeTable fails in an unexpected way

listLeasesForStream

public List<Lease> listLeasesForStream(StreamIdentifier streamIdentifier)
                                throws DependencyException,
                                       InvalidStateException,
                                       ProvisionedThroughputException

List all leases for a given stream synchronously.

Specified by:

listLeasesForStream in interface LeaseRefresher

Returns:

list of leases

Throws:

DependencyException - if DynamoDB scan fails in an unexpected way

InvalidStateException - if lease table does not exist

ProvisionedThroughputException - if DynamoDB scan fails due to lack of capacity

listLeases

public List<Lease> listLeases()
                       throws DependencyException,
                              InvalidStateException,
                              ProvisionedThroughputException

List all objects in table synchronously.

Specified by:

listLeases in interface LeaseRefresher

Returns:

list of leases

Throws:

DependencyException - if DynamoDB scan fails in an unexpected way

InvalidStateException - if lease table does not exist

ProvisionedThroughputException - if DynamoDB scan fails due to lack of capacity

isLeaseTableEmpty

public boolean isLeaseTableEmpty()
                          throws DependencyException,
                                 InvalidStateException,
                                 ProvisionedThroughputException

Check (synchronously) if there are any leases in the lease table.

Specified by:

isLeaseTableEmpty in interface LeaseRefresher

Returns:

true if there are no leases in the lease table

Throws:

DependencyException - if DynamoDB scan fails in an unexpected way

InvalidStateException - if lease table does not exist

ProvisionedThroughputException - if DynamoDB scan fails due to lack of capacity

createLeaseIfNotExists

public boolean createLeaseIfNotExists(@NonNull
                                      @NonNull Lease lease)
                               throws DependencyException,
                                      InvalidStateException,
                                      ProvisionedThroughputException

Create a new lease. Conditional on a lease not already existing with this shardId.

Specified by:

createLeaseIfNotExists in interface LeaseRefresher

Parameters:

lease - the lease to create

Returns:

true if lease was created, false if lease already exists

Throws:

DependencyException - if DynamoDB put fails in an unexpected way

InvalidStateException - if lease table does not exist

ProvisionedThroughputException - if DynamoDB put fails due to lack of capacity

getLease

public Lease getLease(@NonNull
                      @NonNull String leaseKey)
               throws DependencyException,
                      InvalidStateException,
                      ProvisionedThroughputException

Specified by:

getLease in interface LeaseRefresher

Parameters:

leaseKey - Get the lease for this leasekey

Returns:

lease for the specified leaseKey, or null if one doesn't exist

Throws:

DependencyException - if DynamoDB get fails in an unexpected way

InvalidStateException - if lease table does not exist

ProvisionedThroughputException - if DynamoDB get fails due to lack of capacity

renewLease

public boolean renewLease(@NonNull
                          @NonNull Lease lease)
                   throws DependencyException,
                          InvalidStateException,
                          ProvisionedThroughputException

Renew a lease by incrementing the lease counter. Conditional on the leaseCounter in DynamoDB matching the leaseCounter of the input. Mutates the leaseCounter of the passed-in lease object after updating the record in DynamoDB.

Specified by:

renewLease in interface LeaseRefresher

Parameters:

lease - the lease to renew

Returns:

true if renewal succeeded, false otherwise

Throws:

DependencyException - if DynamoDB update fails in an unexpected way

InvalidStateException - if lease table does not exist

ProvisionedThroughputException - if DynamoDB update fails due to lack of capacity

takeLease

public boolean takeLease(@NonNull
                         @NonNull Lease lease,
                         @NonNull
                         @NonNull String owner)
                  throws DependencyException,
                         InvalidStateException,
                         ProvisionedThroughputException

Take a lease for the given owner by incrementing its leaseCounter and setting its owner field. Conditional on the leaseCounter in DynamoDB matching the leaseCounter of the input. Mutates the leaseCounter and owner of the passed-in lease object after updating DynamoDB.

Specified by:

takeLease in interface LeaseRefresher

Parameters:

lease - the lease to take

owner - the new owner

Returns:

true if lease was successfully taken, false otherwise

Throws:

DependencyException - if DynamoDB update fails in an unexpected way

InvalidStateException - if lease table does not exist

ProvisionedThroughputException - if DynamoDB update fails due to lack of capacity

evictLease

public boolean evictLease(@NonNull
                          @NonNull Lease lease)
                   throws DependencyException,
                          InvalidStateException,
                          ProvisionedThroughputException

Evict the current owner of lease by setting owner to null. Conditional on the owner in DynamoDB matching the owner of the input. Mutates the lease counter and owner of the passed-in lease object after updating the record in DynamoDB.

Specified by:

evictLease in interface LeaseRefresher

Parameters:

lease - the lease to void

Returns:

true if eviction succeeded, false otherwise

Throws:

DependencyException - if DynamoDB update fails in an unexpected way

InvalidStateException - if lease table does not exist

ProvisionedThroughputException - if DynamoDB update fails due to lack of capacity

deleteAll

public void deleteAll()
               throws DependencyException,
                      InvalidStateException,
                      ProvisionedThroughputException

Delete all leases from DynamoDB. Useful for tools/utils and testing.

Specified by:

deleteAll in interface LeaseRefresher

Throws:

DependencyException - if DynamoDB scan or delete fail in an unexpected way

InvalidStateException - if lease table does not exist

ProvisionedThroughputException - if DynamoDB scan or delete fail due to lack of capacity

deleteLease

public void deleteLease(@NonNull
                        @NonNull Lease lease)
                 throws DependencyException,
                        InvalidStateException,
                        ProvisionedThroughputException

Delete the given lease from DynamoDB. Does nothing when passed a lease that does not exist in DynamoDB.

Specified by:

deleteLease in interface LeaseRefresher

Parameters:

lease - the lease to delete

Throws:

DependencyException - if DynamoDB delete fails in an unexpected way

InvalidStateException - if lease table does not exist

ProvisionedThroughputException - if DynamoDB delete fails due to lack of capacity

updateLease

public boolean updateLease(@NonNull
                           @NonNull Lease lease)
                    throws DependencyException,
                           InvalidStateException,
                           ProvisionedThroughputException

Update application-specific fields of the given lease in DynamoDB. Does not update fields managed by the leasing library such as leaseCounter, leaseOwner, or leaseKey. Conditional on the leaseCounter in DynamoDB matching the leaseCounter of the input. Increments the lease counter in DynamoDB so that updates can be contingent on other updates. Mutates the lease counter of the passed-in lease object.

Specified by:

updateLease in interface LeaseRefresher

Returns:

true if update succeeded, false otherwise

Throws:

DependencyException - if DynamoDB update fails in an unexpected way

InvalidStateException - if lease table does not exist

ProvisionedThroughputException - if DynamoDB update fails due to lack of capacity

updateLeaseWithMetaInfo

public void updateLeaseWithMetaInfo(Lease lease,
                                    UpdateField updateField)
                             throws DependencyException,
                                    InvalidStateException,
                                    ProvisionedThroughputException

Description copied from interface: LeaseRefresher

Update application-specific fields of the given lease in DynamoDB. Does not update fields managed by the leasing library such as leaseCounter, leaseOwner, or leaseKey.

Specified by:

updateLeaseWithMetaInfo in interface LeaseRefresher

Throws:

DependencyException - if DynamoDB update fails in an unexpected way

InvalidStateException - if lease table does not exist

ProvisionedThroughputException - if DynamoDB update fails due to lack of capacity

getCheckpoint

public ExtendedSequenceNumber getCheckpoint(String leaseKey)
                                     throws ProvisionedThroughputException,
                                            InvalidStateException,
                                            DependencyException

Gets the current checkpoint of the shard. This is useful in the resharding use case where we will wait for the parent shard to complete before starting on the records from a child shard.

Specified by:

getCheckpoint in interface LeaseRefresher

Parameters:

leaseKey - Checkpoint of this shard will be returned

Returns:

Checkpoint of this shard, or null if the shard record doesn't exist.

Throws:

ProvisionedThroughputException - if DynamoDB update fails due to lack of capacity

InvalidStateException - if lease table does not exist

DependencyException - if DynamoDB update fails in an unexpected way

convertAndRethrowExceptions

protected DependencyException convertAndRethrowExceptions(String operation,
                                                          String leaseKey,
                                                          Exception e)
                                                   throws ProvisionedThroughputException,
                                                          InvalidStateException

Throws:

ProvisionedThroughputException

InvalidStateException