Package com.atlassian.crowd.util.persistence.hibernate.batch

Class AbstractBatchProcessor<S>

java.lang.Object
com.atlassian.crowd.util.persistence.hibernate.batch.AbstractBatchProcessor<S>
All Implemented Interfaces:
BatchProcessor<S>
public abstract class AbstractBatchProcessor<S> extends Object implements BatchProcessor<S>
Thread-safe batch processor.

This processor is essentially a heavyweight generic DAO for processing batched operations over a collection of entities.

The batchSize defaults to batchSize and can be manually set (via Spring, for example) using the appropriate setter method. The batchSize should match the hibernate.jdbc.batch_size property defined in the Hibernate configuration.

Each batch operation is first divided into smaller sets of batchSize. If there is an error in processing the batch, the batched JDBC call is rolled-back and the batch is processed individually.

This mechanism ensures very fast (JDBC-batched) inserts and updates and follows it up with a fail-over retry for the failing batches. Callback methods are provided to allow you include the session and transaction management you desire. The processing flow is: 

 call beforeProcessCollection()
 For each item in the collection
     Add the item to the batch collection
     If the batch collection size >= the batch size
         call beforeProcessBatch()
         For each item in the batch collection
            perform the operation on the item
         call afterProcessBatch() or rollbackProcessBatch() if an error occurred
         clear the batch collection
 If there are unprocessed items in the Batch Collection
     call beforeProcessBatch()
     For each item in the batch collection
        perform the operation on the item
     call afterProcessBatch() or rollbackProcessBatch() if an error occurred
 call beforeProcessCollection()
If an error occurred during the processing of a batch collection then: 
 For each item in the batch collection
    call beforeProcessIndividual()
    perform the operation on the item
    call afterProcessIndividual() or rollbackProcessIndividual() if an error occurred

NOTE 1: Do not use this if your database is not transactional, *stab* MySQL ISAM.

Author:
Shihab Hamid, Matthew Jensen

  • Field Details

    • log

      protected final org.slf4j.Logger log

    • batchSize

      protected int batchSize

  • Constructor Details

    • AbstractBatchProcessor

      public AbstractBatchProcessor()

  • Method Details

    • execute

      public final <E> BatchResult<E> execute(HibernateOperation<S> op, Collection<E> objects)
      Specified by:
      execute in interface BatchProcessor<S>

    • execute

      public final <E> BatchResult<E> execute(HibernateOperation<S> op, Collection<E> objects, BulkAuditMapper<E> bulkAuditMapper)
      The set is first divided into smaller sets of batchSize. Each batchSet is added via a batched JDBC call (using Hibernate's batching mechanism). If there is an error in processing the batch, the batched JDBC call is rolled-back and the batchSet is processed individually.

      This mechanism ensures very fast (JDBC-batched) inserts and updates and follows it up with a fail-over retry for the failing batches.

      NOTE: do not use this if your database is not transactional, *stab* MySQL ISAM.

      Specified by:
      execute in interface BatchProcessor<S>
      Parameters:
      op - Hibernate operation to perform (eg. replicate, saveOrUpdate).
      objects - set of DirectoryEntity objects to batch add.
      Returns:
      batch result.

    • auditOperations

      protected abstract void auditOperations(List<AuditLogChangesetEntity> changesetEntities)

    • getSession

      protected abstract S getSession()
      Subclasses should use this method to provide the session to be used with any implementations of HibernateOperation.

    • setBatchSize

      public void setBatchSize(int batchSize)
      The batchSize value should be the same as the hibernate.jdbc.batch_size Hibernate property.
      Parameters:
      batchSize - batch size used to group batches.

    • beforeProcessCollection

      protected abstract void beforeProcessCollection()
      Called before the collection is processed into a batch. Can be used to start session or transaction that will cover the execution of the entire collection which could involve multiple batches.

    • afterProcessCollection

      protected abstract void afterProcessCollection()
      Called when processing the collection has completed successfully.

    • beforeProcessBatch

      protected abstract void beforeProcessBatch()
      Called before processing each batch. Can be used to start session or transaction that will cover this particular batch.

    • afterProcessBatch

      protected abstract void afterProcessBatch()
      Called after successfully processing each batch.

    • rollbackProcessBatch

      protected abstract void rollbackProcessBatch()
      Called after processing each batch where an exception was encountered.

    • beforeProcessIndividual

      protected abstract void beforeProcessIndividual()
      Called before processing an individual item. This method will be called if the batch failed and the processor falls back to processing each item seperately.

    • afterProcessIndividual

      protected abstract void afterProcessIndividual()
      Called after successully processing an item individually.

    • rollbackProcessIndividual

      protected abstract void rollbackProcessIndividual()
      Called after processing an individual item where an exception was encountered.