com.tinkerpop.blueprints.util.wrappers.batch

Class BatchGraph<T extends TransactionalGraph>

java.lang.Object

com.tinkerpop.blueprints.util.wrappers.batch.BatchGraph<T>

All Implemented Interfaces:

Graph, TransactionalGraph, WrapperGraph<T>

public class BatchGraph<T extends TransactionalGraph>
extends Object
implements TransactionalGraph, WrapperGraph<T>

BatchGraph is a wrapper that enables batch loading of a large number of edges and vertices by chunking the entire load into smaller batches and maintaining a memory-efficient vertex cache so that the entire transactional state can be flushed after each chunk is loaded. BatchGraph is ONLY meant for loading data and does not support any retrieval or removal operations. That is, BatchGraph only supports the following methods: - addVertex(Object) for adding vertices - addEdge(Object, com.tinkerpop.blueprints.Vertex, com.tinkerpop.blueprints.Vertex, String) for adding edges - getVertex(Object) to be used when adding edges - Property getter, setter and removal methods for vertices and edges. An important limitation of BatchGraph is that edge properties can only be set immediately after the edge has been added. If other vertices or edges have been created in the meantime, setting, getting or removing properties will throw exceptions. This is done to avoid caching of edges which would require a great amount of memory. BatchGraph wraps TransactionalGraph. To wrap arbitrary graphs, use wrap(com.tinkerpop.blueprints.Graph) which will additionally wrap non-transactional. BatchGraph can also automatically set the provided element ids as properties on the respective element. Use setVertexIdKey(String) and setEdgeIdKey(String) to set the keys for the vertex and edge properties respectively. This allows to make the loaded baseGraph compatible for later wrapping with IdGraph.

Author:

Matthias Broecheler (http://www.matthiasb.com)

Nested Class Summary

Nested classes/interfaces inherited from interface com.tinkerpop.blueprints.TransactionalGraph

TransactionalGraph.Conclusion

Field Summary

Fields

Modifier and Type	Field and Description
static long	DEFAULT_BUFFER_SIZE Default buffer size

Constructor Summary

Constructors

Constructor and Description
BatchGraph(T graph) Constructs a BatchGraph wrapping the provided baseGraph.
BatchGraph(T graph, long bufferSize) Constructs a BatchGraph wrapping the provided baseGraph.
BatchGraph(T graph, VertexIDType type, long bufferSize) Constructs a BatchGraph wrapping the provided baseGraph, using the specified buffer size and expecting vertex ids of the specified IdType.

Method Summary

Methods

Modifier and Type	Method and Description
Edge	addEdge(Object id, Vertex outVertex, Vertex inVertex, String label) Add an edge to the graph.
Edge	addEdge(Object id, Vertex outVertex, Vertex inVertex, String label, Object... properties)
protected Edge	addEdgeSupport(Vertex outVertex, Vertex inVertex, String label)
Vertex	addVertex(Object id) Create a new vertex, add it to the graph, and return the newly created vertex.
Vertex	addVertex(Object id, Object... properties)
void	commit() Should only be invoked after loading is complete.
T	getBaseGraph() Get the graph this wrapper delegates to.
Edge	getEdge(Object id) Return the edge referenced by the provided object identifier.
String	getEdgeIdKey() Returns the key used to set the id on the edges or null if such has not been set via setEdgeIdKey(String)
Iterable<Edge>	getEdges() Return an iterable to all the edges in the graph.
Iterable<Edge>	getEdges(String key, Object value) Return an iterable to all the edges in the graph that have a particular key/value property.
Features	getFeatures() Get the particular features of the graph implementation.
Vertex	getVertex(Object id) If the input data are sorted, then out vertex will be repeated for several edges in a row.
String	getVertexIdKey() Returns the key used to set the id on the vertices or null if such has not been set via setVertexIdKey(String)
Iterable<Vertex>	getVertices() Return an iterable to all the vertices in the graph.
Iterable<Vertex>	getVertices(String key, Object value) Return an iterable to all the vertices in the graph that have a particular key/value property.
boolean	isLoadingFromScratch() Whether this BatchGraph is loading data from scratch or incrementally into an existing graph.
GraphQuery	query() Generate a query object that can be used to fine tune which edges/vertices are retrieved from the graph.
void	removeEdge(Edge edge) Remove the provided edge from the graph.
void	removeVertex(Vertex vertex) Remove the provided vertex from the graph.
void	rollback() Not supported for batch loading, since data may have already been partially persisted.
void	setEdgeIdKey(String key) Sets the key to be used when setting the edge id as a property on the respective edge.
void	setLoadingFromScratch(boolean fromScratch) Sets whether the graph loaded through this instance of BatchGraph is loaded from scratch (i.e.
protected <E extends Element> E	setProperties(E element, Object... properties)
void	setVertexIdKey(String key) Sets the key to be used when setting the vertex id as a property on the respective vertex.
void	shutdown() A shutdown function is required to properly close the graph.
void	stopTransaction(TransactionalGraph.Conclusion conclusion) Should only be invoked after loading is complete.
String	toString()
static BatchGraph	wrap(Graph graph) Constructs a BatchGraph wrapping the provided baseGraph.
static BatchGraph	wrap(Graph graph, long buffer) Constructs a BatchGraph wrapping the provided baseGraph.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

DEFAULT_BUFFER_SIZE

public static final long DEFAULT_BUFFER_SIZE

Default buffer size

See Also:

Constant Field Values

Constructor Detail

BatchGraph

public BatchGraph(T graph,
          VertexIDType type,
          long bufferSize)

Constructs a BatchGraph wrapping the provided baseGraph, using the specified buffer size and expecting vertex ids of the specified IdType. Supplying vertex ids which do not match this type will throw exceptions.

Parameters:

graph - Graph to be wrapped

type - Type of vertex id expected. This information is used to optimize the vertex cache memory footprint.

bufferSize - Defines the number of vertices and edges loaded before starting a new transaction. The larger this value, the more memory is required but the faster the loading process.

BatchGraph

public BatchGraph(T graph,
          long bufferSize)

Constructs a BatchGraph wrapping the provided baseGraph.

Parameters:

graph - Graph to be wrapped

bufferSize - Defines the number of vertices and edges loaded before starting a new transaction. The larger this value, the more memory is required but the faster the loading process.

BatchGraph

public BatchGraph(T graph)

Constructs a BatchGraph wrapping the provided baseGraph.

Parameters:

graph - Graph to be wrapped

Method Detail

wrap

public static BatchGraph wrap(Graph graph)

Constructs a BatchGraph wrapping the provided baseGraph. Immediately returns the baseGraph if its a BatchGraph and wraps non-transactional graphs in an additional WritethroughGraph.

Parameters:

graph - Graph to be wrapped

wrap

public static BatchGraph wrap(Graph graph,
              long buffer)

Constructs a BatchGraph wrapping the provided baseGraph. Immediately returns the baseGraph if its a BatchGraph and wraps non-transactional graphs in an additional WritethroughGraph.

Parameters:

graph - Graph to be wrapped

buffer - Size of the buffer

setVertexIdKey

public void setVertexIdKey(String key)

Sets the key to be used when setting the vertex id as a property on the respective vertex. If the key is null, then no property will be set. If the loaded baseGraph should later be wrapped with IdGraph use IdGraph.ID.

Parameters:

key - Key to be used.

getVertexIdKey

public String getVertexIdKey()

Returns the key used to set the id on the vertices or null if such has not been set via setVertexIdKey(String)

Returns:

The key used to set the id on the vertices or null if such has not been set via setVertexIdKey(String)

setEdgeIdKey

public void setEdgeIdKey(String key)

Sets the key to be used when setting the edge id as a property on the respective edge. If the key is null, then no property will be set. If the loaded baseGraph should later be wrapped with IdGraph use IdGraph.ID.

Parameters:

key - Key to be used.

getEdgeIdKey

public String getEdgeIdKey()

Returns the key used to set the id on the edges or null if such has not been set via setEdgeIdKey(String)

Returns:

The key used to set the id on the edges or null if such has not been set via setEdgeIdKey(String)

setLoadingFromScratch

public void setLoadingFromScratch(boolean fromScratch)

Sets whether the graph loaded through this instance of BatchGraph is loaded from scratch (i.e. the wrapped graph is initially empty) or whether graph is loaded incrementally into an existing graph. In the former case, BatchGraph does not need to check for the existence of vertices with the wrapped graph but only needs to consult its own cache which can be significantly faster. In the latter case, the cache is checked first but an additional check against the wrapped graph may be necessary if the vertex does not exist. By default, BatchGraph assumes that the data is loaded from scratch. When setting loading from scratch to false, a vertex id key must be specified first using setVertexIdKey(String) - otherwise an exception is thrown.

Parameters:

fromScratch -

isLoadingFromScratch

public boolean isLoadingFromScratch()

Whether this BatchGraph is loading data from scratch or incrementally into an existing graph. By default, this returns true.

Returns:

Whether this BatchGraph is loading data from scratch or incrementally into an existing graph.

See Also:

setLoadingFromScratch(boolean)

stopTransaction

public void stopTransaction(TransactionalGraph.Conclusion conclusion)

Should only be invoked after loading is complete. Stopping the transaction before will cause the loading to fail. Only Conclusion.SUCCESS is accepted.

Specified by:

stopTransaction in interface TransactionalGraph

Parameters:

conclusion - whether or not the current transaction was successful

commit

public void commit()

Should only be invoked after loading is complete. Committing the transaction before will cause the loading to fail.

Specified by:

commit in interface TransactionalGraph

rollback

public void rollback()

Not supported for batch loading, since data may have already been partially persisted.

Specified by:

rollback in interface TransactionalGraph

shutdown

public void shutdown()

Description copied from interface: Graph

A shutdown function is required to properly close the graph. This is important for implementations that utilize disk-based serializations.

Specified by:

shutdown in interface Graph

getBaseGraph

public T getBaseGraph()

Description copied from interface: WrapperGraph

Get the graph this wrapper delegates to.

Specified by:

getBaseGraph in interface WrapperGraph<T extends TransactionalGraph>

Returns:

the underlying graph that this WrapperGraph delegates its operations to.

getFeatures

public Features getFeatures()

Description copied from interface: Graph

Get the particular features of the graph implementation. Not all graph implementations are identical nor perfectly implement the Blueprints API. The Features object returned contains meta-data about numerous potential divergences between implementations.

Specified by:

getFeatures in interface Graph

Returns:

the features of this particular Graph implementation

getVertex

public Vertex getVertex(Object id)

If the input data are sorted, then out vertex will be repeated for several edges in a row. In this case, bypass cache and instead immediately return a new vertex using the known id. This gives a modest performance boost, especially when the cache is large or there are on average many edges per vertex.

Specified by:

getVertex in interface Graph

Parameters:

id - the identifier of the vertex to retrieved from the graph

Returns:

the vertex referenced by the provided identifier or null when no such vertex exists

addVertex

public Vertex addVertex(Object id)

Description copied from interface: Graph

Create a new vertex, add it to the graph, and return the newly created vertex. The provided object identifier is a recommendation for the identifier to use. It is not required that the implementation use this identifier.

Specified by:

addVertex in interface Graph

Parameters:

id - the recommended object identifier

Returns:

the newly created vertex

addVertex

public Vertex addVertex(Object id,
               Object... properties)

addEdge

public Edge addEdge(Object id,
           Vertex outVertex,
           Vertex inVertex,
           String label)

Description copied from interface: Graph

Add an edge to the graph. The added edges requires a recommended identifier, a tail vertex, an head vertex, and a label. Like adding a vertex, the provided object identifier may be ignored by the implementation.

Specified by:

addEdge in interface Graph

Parameters:

id - the recommended object identifier

outVertex - the vertex on the tail of the edge

inVertex - the vertex on the head of the edge

label - the label associated with the edge

Returns:

the newly created edge

addEdge

public Edge addEdge(Object id,
           Vertex outVertex,
           Vertex inVertex,
           String label,
           Object... properties)

setProperties

protected <E extends Element> E setProperties(E element,
                                  Object... properties)

addEdgeSupport

protected Edge addEdgeSupport(Vertex outVertex,
                  Vertex inVertex,
                  String label)

toString

public String toString()

Overrides:

toString in class Object

getEdge

public Edge getEdge(Object id)

Description copied from interface: Graph

Return the edge referenced by the provided object identifier. If no edge is referenced by that identifier, then return null.

Specified by:

getEdge in interface Graph

Parameters:

id - the identifier of the edge to retrieved from the graph

Returns:

the edge referenced by the provided identifier or null when no such edge exists

removeVertex

public void removeVertex(Vertex vertex)

Description copied from interface: Graph

Remove the provided vertex from the graph. Upon removing the vertex, all the edges by which the vertex is connected must be removed as well.

Specified by:

removeVertex in interface Graph

Parameters:

vertex - the vertex to remove from the graph

getVertices

public Iterable<Vertex> getVertices()

Description copied from interface: Graph

Return an iterable to all the vertices in the graph. If this is not possible for the implementation, then an UnsupportedOperationException can be thrown.

Specified by:

getVertices in interface Graph

Returns:

an iterable reference to all vertices in the graph

getVertices

public Iterable<Vertex> getVertices(String key,
                           Object value)

Description copied from interface: Graph

Return an iterable to all the vertices in the graph that have a particular key/value property. If this is not possible for the implementation, then an UnsupportedOperationException can be thrown. The graph implementation should use indexing structures to make this efficient else a full vertex-filter scan is required.

Specified by:

getVertices in interface Graph

Parameters:

key - the key of vertex

value - the value of the vertex

Returns:

an iterable of vertices with provided key and value

removeEdge

public void removeEdge(Edge edge)

Description copied from interface: Graph

Remove the provided edge from the graph.

Specified by:

removeEdge in interface Graph

Parameters:

edge - the edge to remove from the graph

getEdges

public Iterable<Edge> getEdges()

Description copied from interface: Graph

Return an iterable to all the edges in the graph. If this is not possible for the implementation, then an UnsupportedOperationException can be thrown.

Specified by:

getEdges in interface Graph

Returns:

an iterable reference to all edges in the graph

getEdges

public Iterable<Edge> getEdges(String key,
                      Object value)

Description copied from interface: Graph

Return an iterable to all the edges in the graph that have a particular key/value property. If this is not possible for the implementation, then an UnsupportedOperationException can be thrown. The graph implementation should use indexing structures to make this efficient else a full edge-filter scan is required.

Specified by:

getEdges in interface Graph

Parameters:

key - the key of the edge

value - the value of the edge

Returns:

an iterable of edges with provided key and value

query

public GraphQuery query()

Description copied from interface: Graph

Generate a query object that can be used to fine tune which edges/vertices are retrieved from the graph.

Specified by:

query in interface Graph

Returns:

a graph query object with methods for constraining which data is pulled from the underlying graph