com.crawljax.core

Class CrawlerExecutor

java.lang.Object

java.util.concurrent.AbstractExecutorService

java.util.concurrent.ThreadPoolExecutor

com.crawljax.core.CrawlerExecutor

All Implemented Interfaces:

Executor, ExecutorService

public class CrawlerExecutor
extends ThreadPoolExecutor

Main Executor inheriting ThreadPoolExecutor to implement the beforeExecute, afterExecute and terminated. Every time a new Crawler is added to this Executor by its ThreadPoolExecutor.execute(Runnable) function a new Thread is created when there are no more free threads. If there is a free (old) unused thread left the Crawler will be loaded into that Thread. So Threads will be reused, at most numberOfThreads will be created. The number of Crawlers active at the same time will be the maximum of the number of Threads. If there are no more Threads left, Crawlers will be stored in a workQueue until a Thread will become available.

Nested Class Summary

Nested classes/interfaces inherited from class java.util.concurrent.ThreadPoolExecutor

ThreadPoolExecutor.AbortPolicy, ThreadPoolExecutor.CallerRunsPolicy, ThreadPoolExecutor.DiscardOldestPolicy, ThreadPoolExecutor.DiscardPolicy

Constructor Summary

Constructors

Constructor and Description
CrawlerExecutor(BrowserConfiguration config) Default CrawlerExecutor.

Method Summary

Methods

Modifier and Type	Method and Description
protected void	afterExecute(Runnable r, Throwable t)
protected void	beforeExecute(Thread t, Runnable r)
boolean	isAborted()
List<Runnable>	shutdownNow(boolean isAbort) shutdown the CrawlerExecutor, depending on the argument issue a abort or normal shutdown.
protected void	terminated()
void	waitForTermination() This function blocks until all tasks has been executed.

Methods inherited from class java.util.concurrent.ThreadPoolExecutor

allowCoreThreadTimeOut, allowsCoreThreadTimeOut, awaitTermination, execute, finalize, getActiveCount, getCompletedTaskCount, getCorePoolSize, getKeepAliveTime, getLargestPoolSize, getMaximumPoolSize, getPoolSize, getQueue, getRejectedExecutionHandler, getTaskCount, getThreadFactory, isShutdown, isTerminated, isTerminating, prestartAllCoreThreads, prestartCoreThread, purge, remove, setCorePoolSize, setKeepAliveTime, setMaximumPoolSize, setRejectedExecutionHandler, setThreadFactory, shutdown, shutdownNow, toString

Methods inherited from class java.util.concurrent.AbstractExecutorService

invokeAll, invokeAll, invokeAny, invokeAny, newTaskFor, newTaskFor, submit, submit, submit

Methods inherited from class java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

CrawlerExecutor

public CrawlerExecutor(BrowserConfiguration config)

Default CrawlerExecutor. using the configured number of threads, no timeout a Stack as workQueue to support Depth-first crawling and the local ThreadFactory. It will have a minimum pool size of the #browsers * 2, and a maximum of #browsers * 5.

Method Detail

afterExecute

protected void afterExecute(Runnable r,
                Throwable t)

Overrides:

afterExecute in class ThreadPoolExecutor

beforeExecute

protected void beforeExecute(Thread t,
                 Runnable r)

Overrides:

beforeExecute in class ThreadPoolExecutor

terminated

protected void terminated()

Overrides:

terminated in class ThreadPoolExecutor

waitForTermination

public void waitForTermination()
                        throws InterruptedException

This function blocks until all tasks has been executed.

Throws:

InterruptedException - when this thread is interrupted.

shutdownNow

public List<Runnable> shutdownNow(boolean isAbort)

shutdown the CrawlerExecutor, depending on the argument issue a abort or normal shutdown.

Parameters:

isAbort - if set to true indicating this is abort instead of normal shutdown.

Returns:

a list of runnables as where in the system on time of the shutdown.

isAborted

public boolean isAborted()

Returns:

true if the shutdown was issues as abort false otherwise.