com.streamsets.pipeline.sdk

Class PushSourceRunner

java.lang.Object

com.streamsets.pipeline.sdk.ProtoRunner

com.streamsets.pipeline.sdk.StageRunner<com.streamsets.pipeline.api.PushSource>

com.streamsets.pipeline.sdk.PushSourceRunner

All Implemented Interfaces:

com.streamsets.datacollector.runner.PushSourceContextDelegate

public class PushSourceRunner
extends StageRunner<com.streamsets.pipeline.api.PushSource>
implements com.streamsets.datacollector.runner.PushSourceContextDelegate

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	PushSourceRunner.Builder
static interface	PushSourceRunner.Callback Interface that caller needs to implement to get produced data.

Nested classes/interfaces inherited from class com.streamsets.pipeline.sdk.StageRunner

StageRunner.Output

Field Summary

Fields inherited from class com.streamsets.pipeline.sdk.ProtoRunner

status

Constructor Summary

Constructors

Constructor and Description
PushSourceRunner(Class<com.streamsets.pipeline.api.PushSource> stageClass, Map<String,Object> configuration, List<String> outputLanes, boolean isPreview, com.streamsets.pipeline.api.OnRecordError onRecordError, Map<String,Object> constants, Map<String,String> stageSdcConf, com.streamsets.pipeline.api.ExecutionMode executionMode, com.streamsets.pipeline.api.DeliveryGuarantee deliveryGuarantee, String resourcesDir, com.streamsets.datacollector.main.RuntimeInfo runtimeInfo, List<ServiceRunner> services)
PushSourceRunner(Class<com.streamsets.pipeline.api.PushSource> stageClass, com.streamsets.pipeline.api.PushSource stage, Map<String,Object> configuration, List<String> outputLanes, boolean isPreview, com.streamsets.pipeline.api.OnRecordError onRecordError, Map<String,Object> constants, Map<String,String> stageSdcConf, com.streamsets.pipeline.api.ExecutionMode executionMode, com.streamsets.pipeline.api.DeliveryGuarantee deliveryGuarantee, String resourcesDir, com.streamsets.datacollector.main.RuntimeInfo runtimeInfo, List<ServiceRunner> services)

Method Summary

Modifier and Type	Method and Description
void	commitOffset(String entityName, String entityOffset)
Map<String,String>	getOffsets() Return offsets that were committed by the origin on last execution.
boolean	processBatch(com.streamsets.pipeline.api.BatchContext batchContext, String entityName, String entityOffset)
void	runProduce(Map<String,String> lastOffsets, int maxBatchSize, PushSourceRunner.Callback callback)
void	setStop() Graceful shutdown.
com.streamsets.pipeline.api.BatchContext	startBatch()
void	waitOnProduce() Wait on the origin's thread to finish.

Methods inherited from class com.streamsets.pipeline.sdk.StageRunner

clearErrors, clearEvents, clearLineageEvents, getContext, getErrorRecords, getErrors, getEventRecords, getInfo, getLineageEvents, getSourceResponseSink, getStage, runDestroy, runInit, runValidateConfigs, setSourceResponseSink

Methods inherited from class com.streamsets.pipeline.sdk.ProtoRunner

configureObject

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PushSourceRunner

public PushSourceRunner(Class<com.streamsets.pipeline.api.PushSource> stageClass,
                        Map<String,Object> configuration,
                        List<String> outputLanes,
                        boolean isPreview,
                        com.streamsets.pipeline.api.OnRecordError onRecordError,
                        Map<String,Object> constants,
                        Map<String,String> stageSdcConf,
                        com.streamsets.pipeline.api.ExecutionMode executionMode,
                        com.streamsets.pipeline.api.DeliveryGuarantee deliveryGuarantee,
                        String resourcesDir,
                        com.streamsets.datacollector.main.RuntimeInfo runtimeInfo,
                        List<ServiceRunner> services)

PushSourceRunner

public PushSourceRunner(Class<com.streamsets.pipeline.api.PushSource> stageClass,
                        com.streamsets.pipeline.api.PushSource stage,
                        Map<String,Object> configuration,
                        List<String> outputLanes,
                        boolean isPreview,
                        com.streamsets.pipeline.api.OnRecordError onRecordError,
                        Map<String,Object> constants,
                        Map<String,String> stageSdcConf,
                        com.streamsets.pipeline.api.ExecutionMode executionMode,
                        com.streamsets.pipeline.api.DeliveryGuarantee deliveryGuarantee,
                        String resourcesDir,
                        com.streamsets.datacollector.main.RuntimeInfo runtimeInfo,
                        List<ServiceRunner> services)

Method Detail

runProduce

public void runProduce(Map<String,String> lastOffsets,
                       int maxBatchSize,
                       PushSourceRunner.Callback callback)
                throws com.streamsets.pipeline.api.StageException

Throws:

com.streamsets.pipeline.api.StageException

setStop

public void setStop()

Graceful shutdown. Will only set flag to stop and return immediately.

waitOnProduce

public void waitOnProduce()
                   throws ExecutionException,
                          InterruptedException

Wait on the origin's thread to finish. Caller code MUST call setStop(), otherwise this method will never return.

Throws:

ExecutionException

InterruptedException

startBatch

public com.streamsets.pipeline.api.BatchContext startBatch()

Specified by:

startBatch in interface com.streamsets.datacollector.runner.PushSourceContextDelegate

processBatch

public boolean processBatch(com.streamsets.pipeline.api.BatchContext batchContext,
                            String entityName,
                            String entityOffset)

Specified by:

processBatch in interface com.streamsets.datacollector.runner.PushSourceContextDelegate

commitOffset

public void commitOffset(String entityName,
                         String entityOffset)

Specified by:

commitOffset in interface com.streamsets.datacollector.runner.PushSourceContextDelegate

getOffsets

public Map<String,String> getOffsets()

Return offsets that were committed by the origin on last execution.