co.cask.cdap.etl.mock.spark.streaming

Class MockSink

java.lang.Object

co.cask.cdap.etl.api.batch.BatchConfigurable<SparkPluginContext>

co.cask.cdap.etl.api.batch.SparkSink<co.cask.cdap.api.data.format.StructuredRecord>

co.cask.cdap.etl.mock.spark.streaming.MockSink

All Implemented Interfaces:

PipelineConfigurable, SubmitterLifecycle<SparkPluginContext>, Serializable

public class MockSink
extends SparkSink<co.cask.cdap.api.data.format.StructuredRecord>

Mock Sink for Spark streaming test.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	MockSink.Config Config for the sink.

Field Summary

Fields

Modifier and Type	Field and Description
static PluginClass	PLUGIN_CLASS

Fields inherited from class co.cask.cdap.etl.api.batch.SparkSink

PLUGIN_TYPE

Constructor Summary

Constructors

Constructor and Description
MockSink(MockSink.Config config)

Method Summary

Modifier and Type	Method and Description
static ETLPlugin	getPlugin(String tableName) Get ETL plugin
static Map<String,String>	getValues(Set<String> keys, DataSetManager<KeyValueTable> tableManager) Get the values associated with the specified keys.
void	prepareRun(SparkPluginContext context)
void	run(SparkExecutionPluginContext context, org.apache.spark.api.java.JavaRDD<co.cask.cdap.api.data.format.StructuredRecord> input)

Methods inherited from class co.cask.cdap.etl.api.batch.BatchConfigurable

configurePipeline, onRunFinish

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PLUGIN_CLASS

public static final PluginClass PLUGIN_CLASS

Constructor Detail

MockSink

public MockSink(MockSink.Config config)

Method Detail

prepareRun

public void prepareRun(SparkPluginContext context)
                throws Exception

Specified by:

prepareRun in interface SubmitterLifecycle<SparkPluginContext>

Specified by:

prepareRun in class BatchConfigurable<SparkPluginContext>

Throws:

Exception

run

public void run(SparkExecutionPluginContext context,
                org.apache.spark.api.java.JavaRDD<co.cask.cdap.api.data.format.StructuredRecord> input)
         throws Exception

Specified by:

run in class SparkSink<co.cask.cdap.api.data.format.StructuredRecord>

Throws:

Exception

getPlugin

public static ETLPlugin getPlugin(String tableName)

Get ETL plugin

Parameters:

tableName - name of the table provided for the configuration

Returns:

an instance of the ETLPlugin

getValues

public static Map<String,String> getValues(Set<String> keys,
                                           DataSetManager<KeyValueTable> tableManager)
                                    throws Exception

Get the values associated with the specified keys.

Parameters:

keys - keys for which value to be determined

tableManager - manager for the table

Returns:

the key value map

Throws:

Exception