org.apache.flink.streaming.api.functions.co

Class CoProcessFunction<IN1,IN2,OUT>

java.lang.Object

org.apache.flink.api.common.functions.AbstractRichFunction

org.apache.flink.streaming.api.functions.co.CoProcessFunction<IN1,IN2,OUT>

Type Parameters:

IN1 - Type of the first input.

IN2 - Type of the second input.

OUT - Output type.

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.functions.Function, org.apache.flink.api.common.functions.RichFunction

@PublicEvolving
public abstract class CoProcessFunction<IN1,IN2,OUT>
extends org.apache.flink.api.common.functions.AbstractRichFunction

A function that processes elements of two streams and produces a single output one.

The function will be called for every element in the input streams and can produce zero or more output elements. Contrary to the CoFlatMapFunction, this function can also query the time (both event and processing) and set timers, through the provided CoProcessFunction.Context. When reacting to the firing of set timers the function can emit yet more elements.

An example use-case for connected streams would be the application of a set of rules that change over time (stream A) to the elements contained in another stream (stream B). The rules contained in stream A can be stored in the state and wait for new elements to arrive on stream B. Upon reception of a new element on stream B, the function can now apply the previously stored rules to the element and directly emit a result, and/or register a timer that will trigger an action in the future.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	CoProcessFunction.Context Information available in an invocation of processElement1(Object, Context, Collector)/ processElement2(Object, Context, Collector) or onTimer(long, OnTimerContext, Collector).
class	CoProcessFunction.OnTimerContext Information available in an invocation of onTimer(long, OnTimerContext, Collector).

Constructor Summary

Constructors

Constructor and Description
CoProcessFunction()

Method Summary

Modifier and Type	Method and Description
void	onTimer(long timestamp, CoProcessFunction.OnTimerContext ctx, org.apache.flink.util.Collector<OUT> out) Called when a timer set using TimerService fires.
abstract void	processElement1(IN1 value, CoProcessFunction.Context ctx, org.apache.flink.util.Collector<OUT> out) This method is called for each element in the first of the connected streams.
abstract void	processElement2(IN2 value, CoProcessFunction.Context ctx, org.apache.flink.util.Collector<OUT> out) This method is called for each element in the second of the connected streams.

Methods inherited from class org.apache.flink.api.common.functions.AbstractRichFunction

close, getIterationRuntimeContext, getRuntimeContext, open, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.api.common.functions.RichFunction

open

Constructor Detail

CoProcessFunction

public CoProcessFunction()

Method Detail

processElement1

public abstract void processElement1(IN1 value,
                                     CoProcessFunction.Context ctx,
                                     org.apache.flink.util.Collector<OUT> out)
                              throws Exception

This method is called for each element in the first of the connected streams.

This function can output zero or more elements using the Collector parameter and also update internal state or set timers using the CoProcessFunction.Context parameter.

Parameters:

value - The stream element

ctx - A CoProcessFunction.Context that allows querying the timestamp of the element, querying the TimeDomain of the firing timer and getting a TimerService for registering timers and querying the time. The context is only valid during the invocation of this method, do not store it.

out - The collector to emit resulting elements to

Throws:

Exception - The function may throw exceptions which cause the streaming program to fail and go into recovery.

processElement2

public abstract void processElement2(IN2 value,
                                     CoProcessFunction.Context ctx,
                                     org.apache.flink.util.Collector<OUT> out)
                              throws Exception

This method is called for each element in the second of the connected streams.

This function can output zero or more elements using the Collector parameter and also update internal state or set timers using the CoProcessFunction.Context parameter.

Parameters:

value - The stream element

ctx - A CoProcessFunction.Context that allows querying the timestamp of the element, querying the TimeDomain of the firing timer and getting a TimerService for registering timers and querying the time. The context is only valid during the invocation of this method, do not store it.

out - The collector to emit resulting elements to

Throws:

Exception - The function may throw exceptions which cause the streaming program to fail and go into recovery.

onTimer

public void onTimer(long timestamp,
                    CoProcessFunction.OnTimerContext ctx,
                    org.apache.flink.util.Collector<OUT> out)
             throws Exception

Called when a timer set using TimerService fires.

Parameters:

timestamp - The timestamp of the firing timer.

ctx - An CoProcessFunction.OnTimerContext that allows querying the timestamp of the firing timer, querying the TimeDomain of the firing timer and getting a TimerService for registering timers and querying the time. The context is only valid during the invocation of this method, do not store it.

out - The collector for returning result values.

Throws:

Exception - This method may throw exceptions. Throwing an exception will cause the operation to fail and may trigger recovery.