com.univocity.parsers.remote
Class RemoteFollower<S extends RemoteEntitySettings,T extends RemoteEntityList<S>,R extends RemoteParserSettings>

java.lang.Object
  com.univocity.parsers.remote.RemoteFollower<S,T,R>

Type Parameters:

S - Type of entity stored in RemoteEntityList associated with the RemoteLinkFollower.

T - The type of RemoteEntityList that contains the entities used to parse a linked page. A RemoteLinkFollower can be thought of as a special type of RemoteEntityList that is used on linked pages.

R - The type of settings that configures a RemoteParser

All Implemented Interfaces:

Cloneable

public abstract class RemoteFollower<S extends RemoteEntitySettings,T extends RemoteEntityList<S>,R extends RemoteParserSettings>
extends Object
implements Cloneable

An abstract class that allow parsers that use RemoteParserSettings and RemoteEntityList to access and parse linked pages. Linked page locations are generated with link following fields based on the content of the page.

Field Summary

protected T	entityList
protected S	entitySettings
protected NextInputHandler<RemoteContext>	nextLinkHandler
protected RemoteFollower	parentLinkFollower
protected R	parserSettings
protected TreeMap<String,com.univocity.api.common.ValueGetter<?>>	urlParameters

Constructor Summary

protected

RemoteFollower(S parentEntitySettings) Creates a new LinkFollower Uses the parentEntitySettings as a basis for the LinkFollower settings

Method Summary

S	addEntity(String entityName) Adds a new entity to this remote follower if it doesn't exist and returns its configuration.
RemoteFollower	assigning(String parameterName, Object parameterValue) Assign the parameter in the Url with the name parameterName to the value parameterValue
RemoteFollower	assigning(String parameterName, com.univocity.api.common.ValueGetter<?> valueGetter) Assign the parameter in the Url with the name ParameterName to the value supplied by the ValueGetter
com.univocity.api.net.UrlReaderProvider	getBaseUrl() Returns the URL the remote follower should work with.
S	getEntity(String entityName) Returns an entity of this remote follower.
T	getEntityList() Returns the list of entities available from this remote follower
Nesting	getNesting() Returns the nesting strategy to apply to rows associated to a "parent" row, such as results parsed from a link accessed by a RemoteFollower.
NextInputHandler<RemoteContext>	getNextLinkHandler() Gets the next link handler.
R	getParserSettings() Returns the settings object associated with the remote follower.
void	ignoreFollowingErrors(boolean ignoreLinkFollowingErrors) Configures the parser to ignore (or not) invalid, malformed or unavailable links when following urls to collect additional data associated to a current result.
boolean	isIgnoreFollowingErrors() Returns a flag indicating whether the parser will ignore invalid, malformed or unavailable links when following urls to collect additional data associated to a current result.
void	setBaseUrl(com.univocity.api.net.UrlReaderProvider baseUrlReaderProvider) Defines a base URL for the remote follower to run.
void	setNesting(Nesting nesting) Configures the nesting strategy to apply to rows associated to a "parent" row, such as results parsed from a link accessed by a RemoteFollower.
void	setNextLinkHandler(NextInputHandler<RemoteContext> nextLinkHandler) Sets the next link handler which will have the NextInputHandler.prepareNextCall(RemoteContext) method called before the next remote page is fetched.
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

entityList

protected T extends RemoteEntityList<S> entityList

parserSettings

protected R extends RemoteParserSettings parserSettings

entitySettings

protected S extends RemoteEntitySettings entitySettings

parentLinkFollower

protected final RemoteFollower parentLinkFollower

nextLinkHandler

protected NextInputHandler<RemoteContext> nextLinkHandler

urlParameters

protected TreeMap<String,com.univocity.api.common.ValueGetter<?>> urlParameters

Constructor Detail

RemoteFollower

protected RemoteFollower(S parentEntitySettings)

Creates a new LinkFollower Uses the parentEntitySettings as a basis for the LinkFollower settings

Parameters:

parentEntitySettings - the parent entity settings used as a basis for LinkFollower settings

Method Detail

getEntityList

public final T getEntityList()

Returns the list of entities available from this remote follower

Returns:

the list of entities available from this remote follower

addEntity

public final S addEntity(String entityName)

Adds a new entity to this remote follower if it doesn't exist and returns its configuration. The global settings made for the parser will be used by default. You can configure your entity to use different settings if required.

Parameters:

entityName - name of the entity whose configuration that will be returned.

Returns:

an existing or new entity configuration associated with the given entity name

getEntity

public final S getEntity(String entityName)

Returns an entity of this remote follower.

Parameters:

entityName - name of the entity whose configuration that will be returned.

Returns:

an existing entity configuration associated with the given entity name

getParserSettings

public final R getParserSettings()

Returns the settings object associated with the remote follower. This configuration object is used to configure the remote follower when it parses a linked page.

Returns:

the remote follower's associated settings object

getBaseUrl

public final com.univocity.api.net.UrlReaderProvider getBaseUrl()

Returns the URL the remote follower should work with. Used for processing links that point to other remote hosts, and to ensure that relative resource paths are resolved against the given base URL.

Returns:

the base URL to resolve the remote address to be accessed by this remote follower.

setBaseUrl

public final void setBaseUrl(com.univocity.api.net.UrlReaderProvider baseUrlReaderProvider)

Defines a base URL for the remote follower to run. Used for processing links that point to other remote hosts, and to ensure that relative resource paths are resolved against the given base URL.

Parameters:

baseUrlReaderProvider - the new baseUrl for processing links.

assigning

public RemoteFollower assigning(String parameterName,
                                Object parameterValue)

Assign the parameter in the Url with the name parameterName to the value parameterValue

Parameters:

parameterName - name of the parameter

parameterValue - value the parameter should hold

Returns:

this RemoteFollower to allow for method chaining

assigning

public RemoteFollower assigning(String parameterName,
                                com.univocity.api.common.ValueGetter<?> valueGetter)

Assign the parameter in the Url with the name ParameterName to the value supplied by the ValueGetter

Parameters:

parameterName - name of the parameter in the Url to assign to

valueGetter - the ValueGetter that will provide the value to be used as the parameter

Returns:

this RemoteFollower to allow for method chaining

ignoreFollowingErrors

public final void ignoreFollowingErrors(boolean ignoreLinkFollowingErrors)

Configures the parser to ignore (or not) invalid, malformed or unavailable links when following urls to collect additional data associated to a current result. If set to false, the parser will throw an Exception when attempting to follow a link that is invalid, malformed or unavailable. If true, the parser will simply ignore the error and proceed. Defaults to true

Parameters:

ignoreLinkFollowingErrors - true if the parser will ignore errors when accessing linked page, false otherwise.

isIgnoreFollowingErrors

public final boolean isIgnoreFollowingErrors()

Returns a flag indicating whether the parser will ignore invalid, malformed or unavailable links when following urls to collect additional data associated to a current result. Defaults to true

Returns:

true if the parser is set to ignore errors when accessing linked page

getNesting

public final Nesting getNesting()

Returns the nesting strategy to apply to rows associated to a "parent" row, such as results parsed from a link accessed by a RemoteFollower. Defaults to the parent entity's RemoteEntitySettings.getNesting() or if undefined, the RemoteParserSettings.getNesting() setting.

Returns:

the nesting strategy to use when processing results associated with a parent row.

setNesting

public final void setNesting(Nesting nesting)

Configures the nesting strategy to apply to rows associated to a "parent" row, such as results parsed from a link accessed by a RemoteFollower. Defaults to the parent entity's RemoteEntitySettings.getNesting() or if undefined, the RemoteParserSettings.getNesting() setting.

Parameters:

nesting - the nesting strategy to use when processing results associated with a parent row.

toString

public String toString()

Overrides:

toString in class Object

getNextLinkHandler

public NextInputHandler<RemoteContext> getNextLinkHandler()

Gets the next link handler. Which will have the NextInputHandler.prepareNextCall(RemoteContext) method called before the next remote page is fetched.

Returns:

the next link handler.

setNextLinkHandler

public void setNextLinkHandler(NextInputHandler<RemoteContext> nextLinkHandler)

Sets the next link handler which will have the NextInputHandler.prepareNextCall(RemoteContext) method called before the next remote page is fetched. For example this could be used to modify the HTTP request configuration.

Parameters:

nextLinkHandler - the new link handler.