org.apache.hadoop.hbase.procedure2

Class Procedure<TEnvironment>

java.lang.Object

org.apache.hadoop.hbase.procedure2.Procedure<TEnvironment>

All Implemented Interfaces:

Comparable<Procedure<TEnvironment>>

Direct Known Subclasses:

OnePhaseProcedure, ProcedureExecutor.FailedProcedure, ProcedureInMemoryChore, SequentialProcedure, StateMachineProcedure, TwoPhaseProcedure

@InterfaceAudience.Private
 @InterfaceStability.Evolving
public abstract class Procedure<TEnvironment>
extends Object
implements Comparable<Procedure<TEnvironment>>

Base Procedure class responsible for Procedure Metadata; e.g. state, submittedTime, lastUpdate, stack-indexes, etc.

Procedures are run by a ProcedureExecutor instance. They are submitted and then the ProcedureExecutor keeps calling execute(Object) until the Procedure is done. Execute may be called multiple times in the case of failure or a restart, so code must be idempotent. The return from an execute call is either: null to indicate we are done; ourself if there is more to do; or, a set of sub-procedures that need to be run to completion before the framework resumes our execution.

The ProcedureExecutor keeps its notion of Procedure State in the Procedure itself; e.g. it stamps the Procedure as INITIALIZING, RUNNABLE, SUCCESS, etc. Here are some of the States defined in the ProcedureState enum from protos:

• isFailed() A procedure has executed at least once and has failed. The procedure may or may not have rolled back yet. Any procedure in FAILED state will be eventually moved to ROLLEDBACK state.
• isSuccess() A procedure is completed successfully without exception.
• isFinished() As a procedure in FAILED state will be tried forever for rollback, only condition when scheduler/ executor will drop procedure from further processing is when procedure state is ROLLEDBACK or isSuccess() returns true. This is a terminal state of the procedure.
• isWaiting() - Procedure is in one of the two waiting states (ProcedureProtos.ProcedureState.WAITING, ProcedureProtos.ProcedureState.WAITING_TIMEOUT).

NOTE: These states are of the ProcedureExecutor. Procedure implementations in turn can keep their own state. This can lead to confusion. Try to keep the two distinct.

rollback() is called when the procedure or one of the sub-procedures has failed. The rollback step is supposed to cleanup the resources created during the execute() step. In case of failure and restart, rollback() may be called multiple times, so again the code must be idempotent.

Procedure can be made respect a locking regime. It has acquire/release methods as well as an hasLock(Object). The lock implementation is up to the implementor. If an entity needs to be locked for the life of a procedure -- not just the calls to execute -- then implementations should say so with the holdLock(Object) method.

Procedures can be suspended or put in wait state with a callback that gets executed on Procedure-specified timeout. See setTimeout(int)}, and setTimeoutFailure(Object). See TestProcedureEvents and the TestTimeoutEventProcedure class for an example usage.

There are hooks for collecting metrics on submit of the procedure and on finish. See updateMetricsOnSubmit(Object) and updateMetricsOnFinish(Object, long, boolean).

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Procedure.LockState

Field Summary

Fields

Modifier and Type	Field and Description
static long	NO_PROC_ID
protected static int	NO_TIMEOUT

Constructor Summary

Constructors

Constructor and Description
Procedure()

Method Summary

Modifier and Type	Method and Description
protected abstract boolean	abort(TEnvironment env) The abort() call is asynchronous and each procedure must decide how to deal with it, if they want to be abortable.
protected Procedure.LockState	acquireLock(TEnvironment env) The user should override this method if they need a lock on an Entity.
protected void	addStackIndex(int index) Called by the RootProcedureState on procedure execution.
protected void	afterReplay(TEnvironment env) Called when the procedure is ready to be added to the queue after the loading/replay operation.
protected void	beforeReplay(TEnvironment env) Called when the procedure is loaded for replay.
int	compareTo(Procedure<TEnvironment> other)
protected void	completionCleanup(TEnvironment env) Called when the procedure is marked as completed (success or rollback).
protected abstract void	deserializeStateData(ProcedureStateSerializer serializer) Called on store load to allow the user to decode the previously serialized state.
protected Procedure.LockState	doAcquireLock(TEnvironment env) Internal method called by the ProcedureExecutor that starts the user-level code acquireLock().
protected Procedure<TEnvironment>[]	doExecute(TEnvironment env) Internal method called by the ProcedureExecutor that starts the user-level code execute().
protected void	doReleaseLock(TEnvironment env) Internal method called by the ProcedureExecutor that starts the user-level code releaseLock().
protected void	doRollback(TEnvironment env) Internal method called by the ProcedureExecutor that starts the user-level code rollback().
long	elapsedTime()
protected abstract Procedure<TEnvironment>[]	execute(TEnvironment env) The main code of the procedure.
protected int	getChildrenLatch()
RemoteProcedureException	getException()
long	getLastUpdate()
NonceKey	getNonceKey()
String	getOwner()
long	getParentProcId()
protected ProcedureMetrics	getProcedureMetrics(TEnvironment env) Override this method to provide procedure specific counters for submitted count, failed count and time histogram.
long	getProcId()
static long	getProcIdHashCode(long procId) Get an hashcode for the specified Procedure ID
String	getProcName()
byte[]	getResult()
protected static Long	getRootProcedureId(Map<Long,Procedure> procedures, Procedure<?> proc)
long	getRootProcId()
protected int[]	getStackIndexes()
org.apache.hadoop.hbase.shaded.protobuf.generated.ProcedureProtos.ProcedureState	getState()
long	getSubmittedTime()
int	getTimeout()
protected long	getTimeoutTimestamp() Timeout of the next timeout.
protected boolean	hasChildren()
boolean	hasException()
protected boolean	hasLock(TEnvironment env) This is used in conjunction with holdLock(Object).
boolean	hasOwner()
boolean	hasParent()
boolean	hasTimeout()
static boolean	haveSameParent(Procedure<?> a, Procedure<?> b)
protected boolean	holdLock(TEnvironment env) Used to keep the procedure lock even when the procedure is yielding or suspended.
protected void	incChildrenLatch() Called by the ProcedureExecutor on procedure-load to restore the latch state
boolean	isFailed()
boolean	isFinished()
boolean	isInitializing()
boolean	isRunnable()
boolean	isSuccess()
boolean	isWaiting()
protected boolean	isYieldAfterExecutionStep(TEnvironment env) By default, the procedure framework/executor will try to run procedures start to finish.
protected void	releaseLock(TEnvironment env) The user should override this method, and release lock if necessary.
protected boolean	removeStackIndex()
protected abstract void	rollback(TEnvironment env) The code to undo what was done by the execute() code.
protected abstract void	serializeStateData(ProcedureStateSerializer serializer) The user-level code of the procedure may have some state to persist (e.g.
protected void	setAbortFailure(String source, String msg)
protected void	setChildrenLatch(int numChildren) Called by the ProcedureExecutor on procedure-load to restore the latch state
protected void	setFailure(RemoteProcedureException exception)
protected void	setFailure(String source, Throwable cause)
protected void	setLastUpdate(long lastUpdate) Called on store load to initialize the Procedure internals after the creation/deserialization.
protected void	setNonceKey(NonceKey nonceKey) Called by the ProcedureExecutor to set the value to the newly created procedure.
void	setOwner(String owner)
void	setOwner(User owner)
protected void	setParentProcId(long parentProcId) Called by the ProcedureExecutor to assign the parent to the newly created procedure.
protected void	setProcId(long procId) Called by the ProcedureExecutor to assign the ID to the newly created procedure.
protected void	setResult(byte[] result) The procedure may leave a "result" on completion.
protected void	setRootProcId(long rootProcId)
protected void	setStackIndexes(List<Integer> stackIndexes) Called on store load to initialize the Procedure internals after the creation/deserialization.
protected void	setState(org.apache.hadoop.hbase.shaded.protobuf.generated.ProcedureProtos.ProcedureState state)
protected void	setSubmittedTime(long submittedTime) Called on store load to initialize the Procedure internals after the creation/deserialization.
protected void	setTimeout(int timeout)
protected boolean	setTimeoutFailure(TEnvironment env) Called by the ProcedureExecutor when the timeout set by setTimeout() is expired.
protected boolean	shouldWaitClientAck(TEnvironment env) By default, the executor will keep the procedure result around util the eviction TTL is expired.
String	toString()
protected String	toStringClass()
protected void	toStringClassDetails(StringBuilder builder) Extend the toString() information with the procedure details e.g.
String	toStringDetails() Extend the toString() information with more procedure details
protected StringBuilder	toStringSimpleSB() Build the StringBuilder for the simple form of procedure string.
protected void	toStringState(StringBuilder builder) Called from toString() when interpolating Procedure State.
protected void	updateMetricsOnFinish(TEnvironment env, long runtime, boolean success) This function will be called just after procedure execution is finished.
protected void	updateMetricsOnSubmit(TEnvironment env) This function will be called just when procedure is submitted for execution.
protected void	updateTimestamp() Called by ProcedureExecutor after each time a procedure step is executed.
protected boolean	wasExecuted()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

NO_PROC_ID

public static final long NO_PROC_ID

See Also:

Constant Field Values

NO_TIMEOUT

protected static final int NO_TIMEOUT

See Also:

Constant Field Values

Constructor Detail

Procedure

public Procedure()

Method Detail

execute

protected abstract Procedure<TEnvironment>[] execute(TEnvironment env)
                                              throws ProcedureYieldException,
                                                     ProcedureSuspendedException,
                                                     InterruptedException

The main code of the procedure. It must be idempotent since execute() may be called multiple times in case of machine failure in the middle of the execution.

Parameters:

env - the environment passed to the ProcedureExecutor

Returns:

a set of sub-procedures to run or ourselves if there is more work to do or null if the procedure is done.

Throws:

ProcedureYieldException - the procedure will be added back to the queue and retried later.

InterruptedException - the procedure will be added back to the queue and retried later.

ProcedureSuspendedException - Signal to the executor that Procedure has suspended itself and has set itself up waiting for an external event to wake it back up again.

rollback

protected abstract void rollback(TEnvironment env)
                          throws IOException,
                                 InterruptedException

The code to undo what was done by the execute() code. It is called when the procedure or one of the sub-procedures failed or an abort was requested. It should cleanup all the resources created by the execute() call. The implementation must be idempotent since rollback() may be called multiple time in case of machine failure in the middle of the execution.

Parameters:

env - the environment passed to the ProcedureExecutor

Throws:

IOException - temporary failure, the rollback will retry later

InterruptedException - the procedure will be added back to the queue and retried later

abort

protected abstract boolean abort(TEnvironment env)

The abort() call is asynchronous and each procedure must decide how to deal with it, if they want to be abortable. The simplest implementation is to have an AtomicBoolean set in the abort() method and then the execute() will check if the abort flag is set or not. abort() may be called multiple times from the client, so the implementation must be idempotent.

NOTE: abort() is not like Thread.interrupt(). It is just a notification that allows the procedure implementor abort.

serializeStateData

protected abstract void serializeStateData(ProcedureStateSerializer serializer)
                                    throws IOException

The user-level code of the procedure may have some state to persist (e.g. input arguments or current position in the processing state) to be able to resume on failure.

Parameters:

serializer - stores the serializable state

Throws:

IOException

deserializeStateData

protected abstract void deserializeStateData(ProcedureStateSerializer serializer)
                                      throws IOException

Called on store load to allow the user to decode the previously serialized state.

Parameters:

serializer - contains the serialized state

Throws:

IOException

acquireLock

protected Procedure.LockState acquireLock(TEnvironment env)

The user should override this method if they need a lock on an Entity. A lock can be anything, and it is up to the implementor. The Procedure Framework will call this method just before it invokes execute(Object). It calls releaseLock(Object) after the call to execute.

If you need to hold the lock for the life of the Procedure -- i.e. you do not want any other Procedure interfering while this Procedure is running, see holdLock(Object).

Example: in our Master we can execute request in parallel for different tables. We can create t1 and create t2 and these creates can be executed at the same time. Anything else on t1/t2 is queued waiting that specific table create to happen.

There are 3 LockState:

• LOCK_ACQUIRED should be returned when the proc has the lock and the proc is ready to execute.
• LOCK_YIELD_WAIT should be returned when the proc has not the lock and the framework should take care of readding the procedure back to the runnable set for retry
• LOCK_EVENT_WAIT should be returned when the proc has not the lock and someone will take care of readding the procedure back to the runnable set when the lock is available.

Returns:

the lock state as described above.

releaseLock

protected void releaseLock(TEnvironment env)

The user should override this method, and release lock if necessary.

holdLock

protected boolean holdLock(TEnvironment env)

Used to keep the procedure lock even when the procedure is yielding or suspended. Must implement hasLock(Object) if you want to hold the lock for life of the Procedure.

Returns:

true if the procedure should hold on the lock until completionCleanup()

See Also:

hasLock(Object)

hasLock

protected boolean hasLock(TEnvironment env)

This is used in conjunction with holdLock(Object). If holdLock(Object) returns true, the procedure executor will call acquireLock() once and thereafter not call releaseLock(Object) until the Procedure is done (Normally, it calls release/acquire around each invocation of execute(Object).

Returns:

true if the procedure has the lock, false otherwise.

See Also:

holdLock(Object)

beforeReplay

protected void beforeReplay(TEnvironment env)

Called when the procedure is loaded for replay. The procedure implementor may use this method to perform some quick operation before replay. e.g. failing the procedure if the state on replay may be unknown.

afterReplay

protected void afterReplay(TEnvironment env)

Called when the procedure is ready to be added to the queue after the loading/replay operation.

completionCleanup

protected void completionCleanup(TEnvironment env)

Called when the procedure is marked as completed (success or rollback). The procedure implementor may use this method to cleanup in-memory states. This operation will not be retried on failure. If a procedure took a lock, it will have been released when this method runs.

isYieldAfterExecutionStep

protected boolean isYieldAfterExecutionStep(TEnvironment env)

By default, the procedure framework/executor will try to run procedures start to finish. Return true to make the executor yield between each execution step to give other procedures a chance to run.

Parameters:

env - the environment passed to the ProcedureExecutor

Returns:

Return true if the executor should yield on completion of an execution step. Defaults to return false.

shouldWaitClientAck

protected boolean shouldWaitClientAck(TEnvironment env)

By default, the executor will keep the procedure result around util the eviction TTL is expired. The client can cut down the waiting time by requesting that the result is removed from the executor. In case of system started procedure, we can force the executor to auto-ack.

Parameters:

env - the environment passed to the ProcedureExecutor

Returns:

true if the executor should wait the client ack for the result. Defaults to return true.

getProcedureMetrics

protected ProcedureMetrics getProcedureMetrics(TEnvironment env)

Override this method to provide procedure specific counters for submitted count, failed count and time histogram.

Parameters:

env - The environment passed to the procedure executor

Returns:

Container object for procedure related metric

updateMetricsOnSubmit

protected void updateMetricsOnSubmit(TEnvironment env)

This function will be called just when procedure is submitted for execution. Override this method to update the metrics at the beginning of the procedure. The default implementation updates submitted counter if getProcedureMetrics(Object) returns non-null ProcedureMetrics.

updateMetricsOnFinish

protected void updateMetricsOnFinish(TEnvironment env,
                                     long runtime,
                                     boolean success)

This function will be called just after procedure execution is finished. Override this method to update metrics at the end of the procedure. If getProcedureMetrics(Object) returns non-null ProcedureMetrics, the default implementation adds runtime of a procedure to a time histogram for successfully completed procedures. Increments failed counter for failed procedures. TODO: As any of the sub-procedures on failure rolls back all procedures in the stack, including successfully finished siblings, this function may get called twice in certain cases for certain procedures. Explore further if this can be called once.

Parameters:

env - The environment passed to the procedure executor

runtime - Runtime of the procedure in milliseconds

success - true if procedure is completed successfully

toString

public String toString()

Overrides:

toString in class Object

toStringSimpleSB

protected StringBuilder toStringSimpleSB()

Build the StringBuilder for the simple form of procedure string.

Returns:

the StringBuilder

toStringDetails

public String toStringDetails()

Extend the toString() information with more procedure details

toStringClass

protected String toStringClass()

toStringState

protected void toStringState(StringBuilder builder)

Called from toString() when interpolating Procedure State. Allows decorating generic Procedure State with Procedure particulars.

Parameters:

builder - Append current ProcedureProtos.ProcedureState

toStringClassDetails

protected void toStringClassDetails(StringBuilder builder)

Extend the toString() information with the procedure details e.g. className and parameters

Parameters:

builder - the string builder to use to append the proc specific information

getProcId

public long getProcId()

hasParent

public boolean hasParent()

getParentProcId

public long getParentProcId()

getRootProcId

public long getRootProcId()

getProcName

public String getProcName()

getNonceKey

public NonceKey getNonceKey()

getSubmittedTime

public long getSubmittedTime()

getOwner

public String getOwner()

hasOwner

public boolean hasOwner()

setProcId

@InterfaceAudience.Private
protected void setProcId(long procId)

Called by the ProcedureExecutor to assign the ID to the newly created procedure.

setParentProcId

@InterfaceAudience.Private
protected void setParentProcId(long parentProcId)

Called by the ProcedureExecutor to assign the parent to the newly created procedure.

setRootProcId

@InterfaceAudience.Private
protected void setRootProcId(long rootProcId)

setNonceKey

@InterfaceAudience.Private
protected void setNonceKey(NonceKey nonceKey)

Called by the ProcedureExecutor to set the value to the newly created procedure.

setOwner

@InterfaceAudience.Private
public void setOwner(String owner)

setOwner

public void setOwner(User owner)

setSubmittedTime

@InterfaceAudience.Private
protected void setSubmittedTime(long submittedTime)

Called on store load to initialize the Procedure internals after the creation/deserialization.

setTimeout

protected void setTimeout(int timeout)

Parameters:

timeout - timeout interval in msec

hasTimeout

public boolean hasTimeout()

getTimeout

public int getTimeout()

Returns:

the timeout in msec

setLastUpdate

@InterfaceAudience.Private
protected void setLastUpdate(long lastUpdate)

Called on store load to initialize the Procedure internals after the creation/deserialization.

updateTimestamp

@InterfaceAudience.Private
protected void updateTimestamp()

Called by ProcedureExecutor after each time a procedure step is executed.

getLastUpdate

public long getLastUpdate()

getTimeoutTimestamp

@InterfaceAudience.Private
protected long getTimeoutTimestamp()

Timeout of the next timeout. Called by the ProcedureExecutor if the procedure has timeout set and the procedure is in the waiting queue.

Returns:

the timestamp of the next timeout.

elapsedTime

public long elapsedTime()

Returns:

the time elapsed between the last update and the start time of the procedure.

getResult

public byte[] getResult()

Returns:

the serialized result if any, otherwise null

setResult

protected void setResult(byte[] result)

The procedure may leave a "result" on completion.

Parameters:

result - the serialized result that will be passed to the client

isRunnable

public boolean isRunnable()

Returns:

true if the procedure is in a RUNNABLE state.

isInitializing

public boolean isInitializing()

isFailed

public boolean isFailed()

Returns:

true if the procedure has failed. It may or may not have rolled back.

isSuccess

public boolean isSuccess()

Returns:

true if the procedure is finished successfully.

isFinished

public boolean isFinished()

Returns:

true if the procedure is finished. The Procedure may be completed successfully or rolledback.

isWaiting

public boolean isWaiting()

Returns:

true if the procedure is waiting for a child to finish or for an external event.

setState

@InterfaceAudience.Private
protected void setState(org.apache.hadoop.hbase.shaded.protobuf.generated.ProcedureProtos.ProcedureState state)

getState

@InterfaceAudience.Private
public org.apache.hadoop.hbase.shaded.protobuf.generated.ProcedureProtos.ProcedureState getState()

setFailure

protected void setFailure(String source,
                          Throwable cause)

setFailure

protected void setFailure(RemoteProcedureException exception)

setAbortFailure

protected void setAbortFailure(String source,
                               String msg)

setTimeoutFailure

protected boolean setTimeoutFailure(TEnvironment env)

Called by the ProcedureExecutor when the timeout set by setTimeout() is expired.

Returns:

true to let the framework handle the timeout as abort, false in case the procedure handled the timeout itself.

hasException

public boolean hasException()

getException

public RemoteProcedureException getException()

setChildrenLatch

@InterfaceAudience.Private
protected void setChildrenLatch(int numChildren)

Called by the ProcedureExecutor on procedure-load to restore the latch state

incChildrenLatch

@InterfaceAudience.Private
protected void incChildrenLatch()

Called by the ProcedureExecutor on procedure-load to restore the latch state

hasChildren

@InterfaceAudience.Private
protected boolean hasChildren()

getChildrenLatch

@InterfaceAudience.Private
protected int getChildrenLatch()

addStackIndex

@InterfaceAudience.Private
protected void addStackIndex(int index)

Called by the RootProcedureState on procedure execution. Each procedure store its stack-index positions.

removeStackIndex

@InterfaceAudience.Private
protected boolean removeStackIndex()

setStackIndexes

@InterfaceAudience.Private
protected void setStackIndexes(List<Integer> stackIndexes)

Called on store load to initialize the Procedure internals after the creation/deserialization.

wasExecuted

@InterfaceAudience.Private
protected boolean wasExecuted()

getStackIndexes

@InterfaceAudience.Private
protected int[] getStackIndexes()

doExecute

@InterfaceAudience.Private
protected Procedure<TEnvironment>[] doExecute(TEnvironment env)
                                                                  throws ProcedureYieldException,
                                                                         ProcedureSuspendedException,
                                                                         InterruptedException

Internal method called by the ProcedureExecutor that starts the user-level code execute().

Throws:

ProcedureSuspendedException - This is used when procedure wants to halt processing and skip out without changing states or releasing any locks held.

ProcedureYieldException

InterruptedException

doRollback

@InterfaceAudience.Private
protected void doRollback(TEnvironment env)
                                              throws IOException,
                                                     InterruptedException

Internal method called by the ProcedureExecutor that starts the user-level code rollback().

Throws:

IOException

InterruptedException

doAcquireLock

@InterfaceAudience.Private
protected Procedure.LockState doAcquireLock(TEnvironment env)

Internal method called by the ProcedureExecutor that starts the user-level code acquireLock().

doReleaseLock

@InterfaceAudience.Private
protected void doReleaseLock(TEnvironment env)

Internal method called by the ProcedureExecutor that starts the user-level code releaseLock().

compareTo

public int compareTo(Procedure<TEnvironment> other)

Specified by:

compareTo in interface Comparable<Procedure<TEnvironment>>

getProcIdHashCode

public static long getProcIdHashCode(long procId)

Get an hashcode for the specified Procedure ID

Returns:

the hashcode for the specified procId

getRootProcedureId

@InterfaceAudience.Private
protected static Long getRootProcedureId(Map<Long,Procedure> procedures,
                                                                    Procedure<?> proc)

haveSameParent

public static boolean haveSameParent(Procedure<?> a,
                                     Procedure<?> b)

Parameters:

a - the first procedure to be compared.

b - the second procedure to be compared.

Returns:

true if the two procedures have the same parent