co.cask.cdap.etl.mock.batch

Class MockRuntimeDatasetSource

java.lang.Object

co.cask.cdap.etl.api.batch.BatchConfigurable<BatchSourceContext>

co.cask.cdap.etl.api.batch.BatchSource<byte[],Row,co.cask.cdap.api.data.format.StructuredRecord>

co.cask.cdap.etl.mock.batch.MockRuntimeDatasetSource

All Implemented Interfaces:

Destroyable, PipelineConfigurable, StageLifecycle<BatchRuntimeContext>, SubmitterLifecycle<BatchSourceContext>, Transformation<KeyValue<byte[],Row>,co.cask.cdap.api.data.format.StructuredRecord>

public class MockRuntimeDatasetSource
extends BatchSource<byte[],Row,co.cask.cdap.api.data.format.StructuredRecord>

Mock source that tests creation of datasets during runtime based on if dataset exists or not.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	MockRuntimeDatasetSource.Config Config for the source.

Field Summary

Fields

Modifier and Type	Field and Description
static PluginClass	PLUGIN_CLASS

Fields inherited from class co.cask.cdap.etl.api.batch.BatchSource

PLUGIN_TYPE

Constructor Summary

Constructors

Constructor and Description
MockRuntimeDatasetSource(MockRuntimeDatasetSource.Config config)

Method Summary

Modifier and Type	Method and Description
void	configurePipeline(PipelineConfigurer pipelineConfigurer)
static ETLPlugin	getPlugin(String tableName, String runtimeDatasetName)
void	initialize(BatchRuntimeContext context)
void	prepareRun(BatchSourceContext context)
void	transform(KeyValue<byte[],Row> input, Emitter<co.cask.cdap.api.data.format.StructuredRecord> emitter)
static void	writeInput(DataSetManager<Table> tableManager, Iterable<co.cask.cdap.api.data.format.StructuredRecord> records) Used to write the input records for the pipeline run.

Methods inherited from class co.cask.cdap.etl.api.batch.BatchSource

destroy

Methods inherited from class co.cask.cdap.etl.api.batch.BatchConfigurable

onRunFinish

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PLUGIN_CLASS

public static final PluginClass PLUGIN_CLASS

Constructor Detail

MockRuntimeDatasetSource

public MockRuntimeDatasetSource(MockRuntimeDatasetSource.Config config)

Method Detail

configurePipeline

public void configurePipeline(PipelineConfigurer pipelineConfigurer)

Specified by:

configurePipeline in interface PipelineConfigurable

Overrides:

configurePipeline in class BatchConfigurable<BatchSourceContext>

initialize

public void initialize(BatchRuntimeContext context)
                throws Exception

Specified by:

initialize in interface StageLifecycle<BatchRuntimeContext>

Overrides:

initialize in class BatchSource<byte[],Row,co.cask.cdap.api.data.format.StructuredRecord>

Throws:

Exception

prepareRun

public void prepareRun(BatchSourceContext context)
                throws Exception

Specified by:

prepareRun in interface SubmitterLifecycle<BatchSourceContext>

Specified by:

prepareRun in class BatchConfigurable<BatchSourceContext>

Throws:

Exception

transform

public void transform(KeyValue<byte[],Row> input,
                      Emitter<co.cask.cdap.api.data.format.StructuredRecord> emitter)
               throws Exception

Specified by:

transform in interface Transformation<KeyValue<byte[],Row>,co.cask.cdap.api.data.format.StructuredRecord>

Overrides:

transform in class BatchSource<byte[],Row,co.cask.cdap.api.data.format.StructuredRecord>

Throws:

Exception

getPlugin

public static ETLPlugin getPlugin(String tableName,
                                  String runtimeDatasetName)

writeInput

public static void writeInput(DataSetManager<Table> tableManager,
                              Iterable<co.cask.cdap.api.data.format.StructuredRecord> records)
                       throws Exception

Used to write the input records for the pipeline run. Should be called after the pipeline has been created.

Parameters:

tableManager - dataset manager used to write to the source dataset

records - records that should be the input for the pipeline

Throws:

Exception