co.cask.cdap.etl.mock.batch.joiner

Class DupeFlagger

java.lang.Object

co.cask.cdap.etl.api.batch.MultiInputBatchConfigurable<BatchJoinerContext>

co.cask.cdap.etl.api.batch.BatchJoiner<co.cask.cdap.api.data.format.StructuredRecord,co.cask.cdap.api.data.format.StructuredRecord,co.cask.cdap.api.data.format.StructuredRecord>

co.cask.cdap.etl.mock.batch.joiner.DupeFlagger

All Implemented Interfaces:

Destroyable, Joiner<co.cask.cdap.api.data.format.StructuredRecord,co.cask.cdap.api.data.format.StructuredRecord,co.cask.cdap.api.data.format.StructuredRecord>, MultiInputPipelineConfigurable, StageLifecycle<BatchJoinerRuntimeContext>, SubmitterLifecycle<BatchJoinerContext>

public class DupeFlagger
extends BatchJoiner<co.cask.cdap.api.data.format.StructuredRecord,co.cask.cdap.api.data.format.StructuredRecord,co.cask.cdap.api.data.format.StructuredRecord>

Plugin that takes exactly 2 inputs and flags records in one input if they also appear in the other.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	DupeFlagger.Config Config for except plugin

Field Summary

Fields

Modifier and Type	Field and Description
static String	NAME
static PluginClass	PLUGIN_CLASS

Fields inherited from class co.cask.cdap.etl.api.batch.BatchJoiner

PLUGIN_TYPE

Constructor Summary

Constructors

Constructor and Description
DupeFlagger(DupeFlagger.Config config)

Method Summary

Modifier and Type	Method and Description
void	configurePipeline(MultiInputPipelineConfigurer pipelineConfigurer)
JoinConfig	getJoinConfig()
static ETLPlugin	getPlugin(String keep, String flagField)
co.cask.cdap.api.data.format.StructuredRecord	joinOn(String stageName, co.cask.cdap.api.data.format.StructuredRecord record)
co.cask.cdap.api.data.format.StructuredRecord	merge(co.cask.cdap.api.data.format.StructuredRecord joinKey, Iterable<JoinElement<co.cask.cdap.api.data.format.StructuredRecord>> joinRow)

Methods inherited from class co.cask.cdap.etl.api.batch.BatchJoiner

destroy, initialize, prepareRun

Methods inherited from class co.cask.cdap.etl.api.batch.MultiInputBatchConfigurable

onRunFinish

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

NAME

public static final String NAME

See Also:

Constant Field Values

PLUGIN_CLASS

public static final PluginClass PLUGIN_CLASS

Constructor Detail

DupeFlagger

public DupeFlagger(DupeFlagger.Config config)

Method Detail

configurePipeline

public void configurePipeline(MultiInputPipelineConfigurer pipelineConfigurer)

Specified by:

configurePipeline in interface MultiInputPipelineConfigurable

Overrides:

configurePipeline in class BatchJoiner<co.cask.cdap.api.data.format.StructuredRecord,co.cask.cdap.api.data.format.StructuredRecord,co.cask.cdap.api.data.format.StructuredRecord>

joinOn

public co.cask.cdap.api.data.format.StructuredRecord joinOn(String stageName,
                                                            co.cask.cdap.api.data.format.StructuredRecord record)
                                                     throws Exception

Throws:

Exception

getJoinConfig

public JoinConfig getJoinConfig()

merge

public co.cask.cdap.api.data.format.StructuredRecord merge(co.cask.cdap.api.data.format.StructuredRecord joinKey,
                                                           Iterable<JoinElement<co.cask.cdap.api.data.format.StructuredRecord>> joinRow)

getPlugin

public static ETLPlugin getPlugin(String keep,
                                  String flagField)