com.amazon.titan.diskstorage.dynamodb.iterator

Class MultiRowParallelScanInterpreter

java.lang.Object

com.amazon.titan.diskstorage.dynamodb.iterator.MultiRowParallelScanInterpreter

All Implemented Interfaces:

ScanContextInterpreter

public class MultiRowParallelScanInterpreter
extends java.lang.Object
implements ScanContextInterpreter

Builds RecordIterators for scan results from a parallel, segmented scan. To do this, this class tracks the current "boundary" keys of each segment. That way, it can avoid returning duplicate keys if a hash key spans multiple segments

Author:

Matthew Sowders, Alexander Patrikalakis, Michael Rodaitis

Constructor Summary

Constructors

Constructor and Description
MultiRowParallelScanInterpreter(DynamoDBStore store, com.thinkaurelius.titan.diskstorage.keycolumnvalue.SliceQuery sliceQuery)

Method Summary

Methods

Modifier and Type	Method and Description
java.util.List<SingleKeyRecordIterator>	buildRecordIterators(ScanContext scanContext) This class relies heavily on the behavior of segmented scans with respect to which hash keys are scanned by each segment.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MultiRowParallelScanInterpreter

public MultiRowParallelScanInterpreter(DynamoDBStore store,
                               com.thinkaurelius.titan.diskstorage.keycolumnvalue.SliceQuery sliceQuery)

Method Detail

buildRecordIterators

public java.util.List<SingleKeyRecordIterator> buildRecordIterators(ScanContext scanContext)

This class relies heavily on the behavior of segmented scans with respect to which hash keys are scanned by each segment. Here's a rough ASCII example to help illustrate: ___________________________ |hk:A |hk:B | ---------------------------- ^segment 1 ^segment 2 Because we are scanning in segments across the entire hash key space, it is possible for the same hash key to appear in two different segments. We are also running all of the scan segments in parallel, so we have no control over which segment returns first. In the example, if segment 2 was the first segment to post a result, we would store hk:B as a "boundary" key. That way when segment 1 eventually reaches hk:B in its scan, we know that another segment has already returned this hash key and we can safely skip returning it. By doing this, we avoid returning a RecordIterator for the same hash key twice and we only need to store at most 2 hash keys per segment.

Specified by:

buildRecordIterators in interface ScanContextInterpreter