com.amazonaws.services.kinesis.leases.impl

Class LeaseTaker<T extends Lease>

java.lang.Object

com.amazonaws.services.kinesis.leases.impl.LeaseTaker<T>

All Implemented Interfaces:

ILeaseTaker<T>

public class LeaseTaker<T extends Lease>
extends Object
implements ILeaseTaker<T>

An implementation of ILeaseTaker that uses DynamoDB via LeaseManager.

Constructor Summary

Constructors

Constructor and Description
LeaseTaker(ILeaseManager<T> leaseManager, String workerIdentifier, long leaseDurationMillis)

Method Summary

Modifier and Type	Method and Description
String	getWorkerIdentifier()
Map<String,T>	takeLeases() Compute the set of leases available to be taken and attempt to take them.
LeaseTaker<T>	withMaxLeasesForWorker(int maxLeasesForWorker) Worker will not acquire more than the specified max number of leases even if there are more shards that need to be processed.
LeaseTaker<T>	withMaxLeasesToStealAtOneTime(int maxLeasesToStealAtOneTime) Max leases to steal from a more loaded Worker at one time (for load balancing).

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LeaseTaker

public LeaseTaker(ILeaseManager<T> leaseManager,
                  String workerIdentifier,
                  long leaseDurationMillis)

Method Detail

withMaxLeasesForWorker

public LeaseTaker<T> withMaxLeasesForWorker(int maxLeasesForWorker)

Worker will not acquire more than the specified max number of leases even if there are more shards that need to be processed. This can be used in scenarios where a worker is resource constrained or to prevent lease thrashing when small number of workers pick up all leases for small amount of time during deployment. Note that setting a low value may cause data loss (e.g. if there aren't enough Workers to make progress on all shards). When setting the value for this property, one must ensure enough workers are present to process shards and should consider future resharding, child shards that may be blocked on parent shards, some workers becoming unhealthy, etc.

Parameters:

maxLeasesForWorker - Max leases this Worker can handle at a time

Returns:

LeaseTaker

withMaxLeasesToStealAtOneTime

public LeaseTaker<T> withMaxLeasesToStealAtOneTime(int maxLeasesToStealAtOneTime)

Max leases to steal from a more loaded Worker at one time (for load balancing). Setting this to a higher number can allow for faster load convergence (e.g. during deployments, cold starts), but can cause higher churn in the system.

Parameters:

maxLeasesToStealAtOneTime - Steal up to this many leases at one time (for load balancing)

Returns:

LeaseTaker

takeLeases

public Map<String,T> takeLeases()
                         throws DependencyException,
                                InvalidStateException

Compute the set of leases available to be taken and attempt to take them. Lease taking rules are: 1) If a lease's counter hasn't changed in long enough, try to take it. 2) If we see a lease we've never seen before, take it only if owner == null. If it's owned, odds are the owner is holding it. We can't tell until we see it more than once. 3) For load balancing purposes, you may violate rules 1 and 2 for EXACTLY ONE lease per call of takeLeases().

Specified by:

takeLeases in interface ILeaseTaker<T extends Lease>

Returns:

map of shardId to Lease object for leases we just successfully took.

Throws:

DependencyException - on unexpected DynamoDB failures

InvalidStateException - if lease table does not exist

getWorkerIdentifier

public String getWorkerIdentifier()

Specified by:

getWorkerIdentifier in interface ILeaseTaker<T extends Lease>

Returns:

workerIdentifier for this LeaseTaker