com.crawljax.test

Class BaseCrawler

java.lang.Object

com.crawljax.test.BaseCrawler

public class BaseCrawler
extends Object

Base class for Crawler examples.

Constructor Summary

Constructors

Constructor and Description
BaseCrawler(org.eclipse.jetty.util.resource.Resource webfolder)
BaseCrawler(org.eclipse.jetty.util.resource.Resource webfolder, String siteExtension)
BaseCrawler(String siteExtension)

Method Summary

Methods

Modifier and Type	Method and Description
com.crawljax.core.CrawlSession	crawl() Runs the crawler.
com.crawljax.core.CrawlSession	crawlWith(com.crawljax.core.plugin.Plugin... plugins) Runs a crawl with the given plugins.
com.crawljax.core.configuration.CrawljaxConfiguration	getConfig()
protected URI	getUrl()
WebServer	getWebServer()
protected com.crawljax.core.configuration.CrawljaxConfiguration.CrawljaxConfigurationBuilder	newCrawlConfigurationBuilder() Override this method to specify a different configuration.
void	setup() Starts the webserver, configures crawljax.
void	showWebSite() This starts the webserver and blocks the thread so you can inspect the site in your browser manually.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BaseCrawler

public BaseCrawler(String siteExtension)

Parameters:

siteExtension - Assumes the sites are in resources/sites.

BaseCrawler

public BaseCrawler(org.eclipse.jetty.util.resource.Resource webfolder,
           String siteExtension)

Parameters:

webfolder - The folder the web site is stored in.

siteExtension - The extention of the site. Leave blank or null for no extention.

BaseCrawler

public BaseCrawler(org.eclipse.jetty.util.resource.Resource webfolder)

Parameters:

webfolder - The folder the web site is stored in.

Method Detail

getWebServer

public WebServer getWebServer()

setup

public void setup()

Starts the webserver, configures crawljax. Doesn't run the Crawler. For that you need to call crawl(). After this method completes, you can configure the CrawljaxConfiguration and the CrawljaxConfiguration using newCrawlConfigurationBuilder();

The CrawljaxConfiguration is configured with CrawljaxConfiguration.CrawljaxConfigurationBuilder.

Throws:

Exception - When the webserver fails to start.

newCrawlConfigurationBuilder

protected com.crawljax.core.configuration.CrawljaxConfiguration.CrawljaxConfigurationBuilder newCrawlConfigurationBuilder()

Override this method to specify a different configuration.

Returns:

a new CrawljaxConfiguration to crawl with.

getUrl

protected URI getUrl()

getConfig

public com.crawljax.core.configuration.CrawljaxConfiguration getConfig()

Returns:

the CrawljaxConfiguration or null if setup() or crawl() haven't been called yet.

crawl

public com.crawljax.core.CrawlSession crawl()
                                     throws com.crawljax.core.CrawljaxException

Runs the crawler. If you haven't run setup() it will do that before it runs.

Returns:

CrawlSession for post-crawl inspection.

Throws:

Exception - When crawljax isn't configured correctly or the webserver fails to stop.

com.crawljax.core.CrawljaxException

crawlWith

public com.crawljax.core.CrawlSession crawlWith(com.crawljax.core.plugin.Plugin... plugins)

Runs a crawl with the given plugins.

Parameters:

plugins - The plugins you want to run with the CrawljaxConfiguration.

Returns:

The resulting CrawlSession.

See Also:

crawl()

showWebSite

public void showWebSite()
                 throws Exception

This starts the webserver and blocks the thread so you can inspect the site in your browser manually. The URL to visit is printed to the console.

This thread will block once started so you have to stop it manually, once you are done.

Throws:

Exception - in case the webserver can't start