WALProcedureStore (Apache HBase - Procedure 2.0.0 API)

java.lang.Object
- org.apache.hadoop.hbase.procedure2.store.ProcedureStoreBase
- - org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore

All Implemented Interfaces:: ProcedureStore

@InterfaceAudience.Private
 @InterfaceStability.Evolving
public class WALProcedureStore
extends ProcedureStoreBase

WAL implementation of the ProcedureStore.

Nested Class Summary

Nested Classes
Modifier and Type Class and Description

static interface WALProcedureStore.LeaseRecovery

static class WALProcedureStore.SyncMetrics
- Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.procedure2.store.ProcedureStore
  ProcedureStore.ProcedureIterator, ProcedureStore.ProcedureLoader, ProcedureStore.ProcedureStoreListener

Nested Classes
Modifier and Type	Class and Description
`static interface`	`WALProcedureStore.LeaseRecovery`
`static class`	`WALProcedureStore.SyncMetrics`

Field Summary

Fields
Modifier and Type	Field and Description
`static String`	`EXEC_WAL_CLEANUP_ON_LOAD_CONF_KEY`
`static String`	`LOG_PREFIX`
`static String`	`MASTER_PROCEDURE_LOGDIR` Used to construct the name of the log directory for master procedures
`static String`	`MAX_RETRIES_BEFORE_ROLL_CONF_KEY`
`static String`	`MAX_SYNC_FAILURE_ROLL_CONF_KEY`
`static String`	`PERIODIC_ROLL_CONF_KEY`
`static String`	`ROLL_RETRIES_CONF_KEY`
`static String`	`ROLL_THRESHOLD_CONF_KEY`
`static String`	`STORE_WAL_SYNC_STATS_COUNT`
`static String`	`SYNC_WAIT_MSEC_CONF_KEY`
`static String`	`USE_HSYNC_CONF_KEY`
`static String`	`WAIT_BEFORE_ROLL_CONF_KEY`
`static String`	`WAL_COUNT_WARN_THRESHOLD_CONF_KEY`

Constructor Summary

Constructors
Constructor and Description
`WALProcedureStore(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path walDir, org.apache.hadoop.fs.Path walArchiveDir, WALProcedureStore.LeaseRecovery leaseRecovery)`
`WALProcedureStore(org.apache.hadoop.conf.Configuration conf, WALProcedureStore.LeaseRecovery leaseRecovery)`

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`void`	`delete(long procId)` The specified procId was removed from the executor, due to completion, abort or failure.
`void`	`delete(long[] procIds, int offset, int count)` The specified procIds were removed from the executor, due to completion, abort or failure.
`void`	`delete(Procedure proc, long[] subProcIds)` The parent procedure completed.
`ArrayList<ProcedureWALFile>`	`getActiveLogs()`
`Set<ProcedureWALFile>`	`getCorruptedLogs()`
`org.apache.hadoop.fs.FileSystem`	`getFileSystem()`
`protected org.apache.hadoop.fs.Path`	`getLogFilePath(long logId)`
`long`	`getMillisFromLastRoll()`
`long`	`getMillisToNextPeriodicRoll()`
`int`	`getNumThreads()`
`ProcedureStoreTracker`	`getStoreTracker()`
`ArrayList<WALProcedureStore.SyncMetrics>`	`getSyncMetrics()`
`org.apache.hadoop.fs.Path`	`getWALDir()`
`void`	`insert(Procedure[] procs)` Serialize a set of new procedures.
`void`	`insert(Procedure proc, Procedure[] subprocs)` When a procedure is submitted to the executor insert(proc, null) will be called.
`void`	`load(ProcedureStore.ProcedureLoader loader)` Load the Procedures in the store.
`protected void`	`periodicRollForTesting()`
`void`	`recoverLease()` Acquire the lease for the procedure store.
`protected void`	`removeInactiveLogsForTesting()`
`protected boolean`	`rollWriterForTesting()`
`int`	`setRunningProcedureCount(int count)` Set the number of procedure running.
`void`	`start(int numSlots)` Start/Open the procedure store
`void`	`stop(boolean abort)` Stop/Close the procedure store
`protected long`	`syncSlots(org.apache.hadoop.fs.FSDataOutputStream stream, ByteSlot[] slots, int offset, int count)`
`protected void`	`syncStream(org.apache.hadoop.fs.FSDataOutputStream stream)`
`void`	`update(Procedure proc)` The specified procedure was executed, and the new state should be written to the store.

Methods inherited from class org.apache.hadoop.hbase.procedure2.store.ProcedureStoreBase
isRunning, registerListener, sendAbortProcessSignal, sendPostSyncSignal, setRunning, unregisterListener

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - LOG_PREFIX
```
public static final String LOG_PREFIX
```
    See Also:
    
    Constant Field Values
  - MASTER_PROCEDURE_LOGDIR
```
public static final String MASTER_PROCEDURE_LOGDIR
```
    Used to construct the name of the log directory for master procedures
    
    See Also:
    
    Constant Field Values
  - WAL_COUNT_WARN_THRESHOLD_CONF_KEY
```
public static final String WAL_COUNT_WARN_THRESHOLD_CONF_KEY
```
    See Also:
    
    Constant Field Values
  - EXEC_WAL_CLEANUP_ON_LOAD_CONF_KEY
```
public static final String EXEC_WAL_CLEANUP_ON_LOAD_CONF_KEY
```
    See Also:
    
    Constant Field Values
  - MAX_RETRIES_BEFORE_ROLL_CONF_KEY
```
public static final String MAX_RETRIES_BEFORE_ROLL_CONF_KEY
```
    See Also:
    
    Constant Field Values
  - WAIT_BEFORE_ROLL_CONF_KEY
```
public static final String WAIT_BEFORE_ROLL_CONF_KEY
```
    See Also:
    
    Constant Field Values
  - ROLL_RETRIES_CONF_KEY
```
public static final String ROLL_RETRIES_CONF_KEY
```
    See Also:
    
    Constant Field Values
  - MAX_SYNC_FAILURE_ROLL_CONF_KEY
```
public static final String MAX_SYNC_FAILURE_ROLL_CONF_KEY
```
    See Also:
    
    Constant Field Values
  - PERIODIC_ROLL_CONF_KEY
```
public static final String PERIODIC_ROLL_CONF_KEY
```
    See Also:
    
    Constant Field Values
  - SYNC_WAIT_MSEC_CONF_KEY
```
public static final String SYNC_WAIT_MSEC_CONF_KEY
```
    See Also:
    
    Constant Field Values
  - USE_HSYNC_CONF_KEY
```
public static final String USE_HSYNC_CONF_KEY
```
    See Also:
    
    Constant Field Values
  - ROLL_THRESHOLD_CONF_KEY
```
public static final String ROLL_THRESHOLD_CONF_KEY
```
    See Also:
    
    Constant Field Values
  - STORE_WAL_SYNC_STATS_COUNT
```
public static final String STORE_WAL_SYNC_STATS_COUNT
```
    See Also:
    
    Constant Field Values
- Constructor Detail
  - WALProcedureStore
```
public WALProcedureStore(org.apache.hadoop.conf.Configuration conf,
                         WALProcedureStore.LeaseRecovery leaseRecovery)
                  throws IOException
```
    Throws:
    
    IOException
  - WALProcedureStore
```
public WALProcedureStore(org.apache.hadoop.conf.Configuration conf,
                         org.apache.hadoop.fs.Path walDir,
                         org.apache.hadoop.fs.Path walArchiveDir,
                         WALProcedureStore.LeaseRecovery leaseRecovery)
                  throws IOException
```
    Throws:
    
    IOException
- Method Detail
  - start
```
public void start(int numSlots)
           throws IOException
```
    Description copied from interface: ProcedureStore
    
    Start/Open the procedure store
    
    Throws:
    
    IOException
  - stop
```
public void stop(boolean abort)
```
    Description copied from interface: ProcedureStore
    
    Stop/Close the procedure store
    
    Parameters:
    
    abort - true if the stop is an abort
  - getNumThreads
```
public int getNumThreads()
```
    Returns:
    
    the number of threads/slots passed to start()
  - setRunningProcedureCount
```
public int setRunningProcedureCount(int count)
```
    Description copied from interface: ProcedureStore
    
    Set the number of procedure running. This can be used, for example, by the store to know how long to wait before a sync.
    
    Returns:
    
    how many procedures are running (may not be same as count).
  - getStoreTracker
```
public ProcedureStoreTracker getStoreTracker()
```
  - getActiveLogs
```
public ArrayList<ProcedureWALFile> getActiveLogs()
```
  - getCorruptedLogs
```
public Set<ProcedureWALFile> getCorruptedLogs()
```
  - recoverLease
```
public void recoverLease()
                  throws IOException
```
    Description copied from interface: ProcedureStore
    
    Acquire the lease for the procedure store.
    
    Throws:
    
    IOException
  - load
```
public void load(ProcedureStore.ProcedureLoader loader)
          throws IOException
```
    Description copied from interface: ProcedureStore
    
    Load the Procedures in the store.
    
    Parameters:
    
    loader - the ProcedureLoader that will handle the store-load events
    
    Throws:
    
    IOException
  - insert
```
public void insert(Procedure proc,
                   Procedure[] subprocs)
```
    Description copied from interface: ProcedureStore
    
    When a procedure is submitted to the executor insert(proc, null) will be called. 'proc' has a 'RUNNABLE' state and the initial information required to start up. When a procedure is executed and it returns children insert(proc, subprocs) will be called. 'proc' has a 'WAITING' state and an update state. 'subprocs' are the children in 'RUNNABLE' state with the initial information.
    
    Parameters:
    
    proc - the procedure to serialize and write to the store.
    
    subprocs - the newly created child of the proc.
  - insert
```
public void insert(Procedure[] procs)
```
    Description copied from interface: ProcedureStore
    
    Serialize a set of new procedures. These procedures are freshly submitted to the executor and each procedure has a 'RUNNABLE' state and the initial information required to start up.
    
    Parameters:
    
    procs - the procedures to serialize and write to the store.
  - update
```
public void update(Procedure proc)
```
    Description copied from interface: ProcedureStore
    
    The specified procedure was executed, and the new state should be written to the store.
    
    Parameters:
    
    proc - the procedure to serialize and write to the store.
  - delete
```
public void delete(long procId)
```
    Description copied from interface: ProcedureStore
    
    The specified procId was removed from the executor, due to completion, abort or failure. The store implementor should remove all the information about the specified procId.
    
    Parameters:
    
    procId - the ID of the procedure to remove.
  - delete
```
public void delete(Procedure proc,
                   long[] subProcIds)
```
    Description copied from interface: ProcedureStore
    
    The parent procedure completed. Update the state and mark all the child deleted.
    
    Parameters:
    
    proc - the parent procedure to serialize and write to the store.
    
    subProcIds - the IDs of the sub-procedure to remove.
  - delete
```
public void delete(long[] procIds,
                   int offset,
                   int count)
```
    Description copied from interface: ProcedureStore
    
    The specified procIds were removed from the executor, due to completion, abort or failure. The store implementor should remove all the information about the specified procIds.
    
    Parameters:
    
    procIds - the IDs of the procedures to remove.
    
    offset - the array offset from where to start to delete
    
    count - the number of IDs to delete
  - getSyncMetrics
```
public ArrayList<WALProcedureStore.SyncMetrics> getSyncMetrics()
```
  - syncSlots
```
protected long syncSlots(org.apache.hadoop.fs.FSDataOutputStream stream,
                         ByteSlot[] slots,
                         int offset,
                         int count)
                  throws IOException
```
    Throws:
    
    IOException
  - syncStream
```
protected void syncStream(org.apache.hadoop.fs.FSDataOutputStream stream)
                   throws IOException
```
    Throws:
    
    IOException
  - getMillisToNextPeriodicRoll
```
public long getMillisToNextPeriodicRoll()
```
  - getMillisFromLastRoll
```
public long getMillisFromLastRoll()
```
  - periodicRollForTesting
```
protected void periodicRollForTesting()
                               throws IOException
```
    Throws:
    
    IOException
  - rollWriterForTesting
```
protected boolean rollWriterForTesting()
                                throws IOException
```
    Throws:
    
    IOException
  - removeInactiveLogsForTesting
```
protected void removeInactiveLogsForTesting()
                                     throws Exception
```
    Throws:
    
    Exception
  - getWALDir
```
public org.apache.hadoop.fs.Path getWALDir()
```
  - getFileSystem
```
public org.apache.hadoop.fs.FileSystem getFileSystem()
```
  - getLogFilePath
```
protected org.apache.hadoop.fs.Path getLogFilePath(long logId)
                                            throws IOException
```
    Throws:
    
    IOException

Class WALProcedureStore

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.procedure2.store.ProcedureStore

Field Summary

Constructor Summary

Method Summary

Methods inherited from class org.apache.hadoop.hbase.procedure2.store.ProcedureStoreBase

Methods inherited from class java.lang.Object

Field Detail

LOG_PREFIX

MASTER_PROCEDURE_LOGDIR

WAL_COUNT_WARN_THRESHOLD_CONF_KEY

EXEC_WAL_CLEANUP_ON_LOAD_CONF_KEY

MAX_RETRIES_BEFORE_ROLL_CONF_KEY

WAIT_BEFORE_ROLL_CONF_KEY

ROLL_RETRIES_CONF_KEY

MAX_SYNC_FAILURE_ROLL_CONF_KEY

PERIODIC_ROLL_CONF_KEY

SYNC_WAIT_MSEC_CONF_KEY

USE_HSYNC_CONF_KEY

ROLL_THRESHOLD_CONF_KEY

STORE_WAL_SYNC_STATS_COUNT

Constructor Detail

WALProcedureStore

WALProcedureStore

Method Detail

start

stop

getNumThreads

setRunningProcedureCount

getStoreTracker

getActiveLogs

getCorruptedLogs

recoverLease

load

insert

insert

update

delete

delete

delete

getSyncMetrics

syncSlots

syncStream

getMillisToNextPeriodicRoll

getMillisFromLastRoll

periodicRollForTesting

rollWriterForTesting

removeInactiveLogsForTesting

getWALDir

getFileSystem

getLogFilePath