co.cask.cdap.etl.mock.batch

Class MockExternalSource

java.lang.Object

co.cask.cdap.etl.api.batch.BatchConfigurable<BatchSourceContext>

co.cask.cdap.etl.api.batch.BatchSource<LongWritable,Text,co.cask.cdap.api.data.format.StructuredRecord>

co.cask.cdap.etl.mock.batch.MockExternalSource

All Implemented Interfaces:

Destroyable, PipelineConfigurable, StageLifecycle<BatchRuntimeContext>, SubmitterLifecycle<BatchSourceContext>, Transformation<KeyValue<LongWritable,Text>,co.cask.cdap.api.data.format.StructuredRecord>

public class MockExternalSource
extends BatchSource<LongWritable,Text,co.cask.cdap.api.data.format.StructuredRecord>

Mock source that can be used to write a list of records in a Table and reads them out in a pipeline run.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	MockExternalSource.Config Config for the source.

Field Summary

Fields

Modifier and Type	Field and Description
static PluginClass	PLUGIN_CLASS
static String	PLUGIN_NAME

Fields inherited from class co.cask.cdap.etl.api.batch.BatchSource

PLUGIN_TYPE

Constructor Summary

Constructors

Constructor and Description
MockExternalSource(MockExternalSource.Config config)

Method Summary

Modifier and Type	Method and Description
static ETLPlugin	getPlugin(String name, String dirName)
void	prepareRun(BatchSourceContext context)
void	transform(KeyValue<LongWritable,Text> input, Emitter<co.cask.cdap.api.data.format.StructuredRecord> emitter)
static void	writeInput(String fileName, Iterable<co.cask.cdap.api.data.format.StructuredRecord> records) Used to write the input records for the pipeline run.

Methods inherited from class co.cask.cdap.etl.api.batch.BatchSource

destroy, initialize

Methods inherited from class co.cask.cdap.etl.api.batch.BatchConfigurable

configurePipeline, onRunFinish

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PLUGIN_CLASS

public static final PluginClass PLUGIN_CLASS

PLUGIN_NAME

public static final String PLUGIN_NAME

See Also:

Constant Field Values

Constructor Detail

MockExternalSource

public MockExternalSource(MockExternalSource.Config config)

Method Detail

transform

public void transform(KeyValue<LongWritable,Text> input,
                      Emitter<co.cask.cdap.api.data.format.StructuredRecord> emitter)
               throws Exception

Specified by:

transform in interface Transformation<KeyValue<LongWritable,Text>,co.cask.cdap.api.data.format.StructuredRecord>

Overrides:

transform in class BatchSource<LongWritable,Text,co.cask.cdap.api.data.format.StructuredRecord>

Throws:

Exception

prepareRun

public void prepareRun(BatchSourceContext context)
                throws Exception

Specified by:

prepareRun in interface SubmitterLifecycle<BatchSourceContext>

Specified by:

prepareRun in class BatchConfigurable<BatchSourceContext>

Throws:

Exception

getPlugin

public static ETLPlugin getPlugin(String name,
                                  String dirName)

writeInput

public static void writeInput(String fileName,
                              Iterable<co.cask.cdap.api.data.format.StructuredRecord> records)
                       throws Exception

Used to write the input records for the pipeline run. Should be called after the pipeline has been created.

Parameters:

fileName - file to write the records into

records - records that should be the input for the pipeline

Throws:

Exception