Package org.apache.beam.fn.harness

Class PrecombineGroupingTable<K,InputT,AccumT>

java.lang.Object

org.apache.beam.fn.harness.PrecombineGroupingTable<K,InputT,AccumT>

All Implemented Interfaces:

Cache.Shrinkable<PrecombineGroupingTable<K,InputT,AccumT>>, org.apache.beam.sdk.util.Weighted

@NotThreadSafe
public class PrecombineGroupingTable<K,InputT,AccumT>
extends java.lang.Object
implements Cache.Shrinkable<PrecombineGroupingTable<K,InputT,AccumT>>, org.apache.beam.sdk.util.Weighted

Static utility methods that provide a grouping table implementation.

NotThreadSafe because the caller must use the bundle processing thread when invoking put(org.apache.beam.sdk.values.WindowedValue<org.apache.beam.sdk.values.KV<K, InputT>>, org.apache.beam.sdk.fn.data.FnDataReceiver<org.apache.beam.sdk.values.WindowedValue<org.apache.beam.sdk.values.KV<K, AccumT>>>) and flush(org.apache.beam.sdk.fn.data.FnDataReceiver<org.apache.beam.sdk.values.WindowedValue<org.apache.beam.sdk.values.KV<K, AccumT>>>). shrink() may be called from any thread.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static interface	PrecombineGroupingTable.SizeEstimator	Provides client-specific operations for size estimates.

Method Summary

Modifier and Type	Method	Description
static <K,InputT,AccumT> PrecombineGroupingTable<K,InputT,AccumT>	combining(org.apache.beam.sdk.options.PipelineOptions options, Cache<java.lang.Object,java.lang.Object> cache, org.apache.beam.sdk.transforms.Combine.CombineFn<InputT,AccumT,?> combineFn, org.apache.beam.sdk.coders.Coder<K> keyCoder, boolean isGloballyWindowed)	Returns a grouping table that combines inputs into an accumulator.
static <K,InputT,AccumT> PrecombineGroupingTable<K,InputT,AccumT>	combiningAndSampling(org.apache.beam.sdk.options.PipelineOptions options, Cache<java.lang.Object,java.lang.Object> cache, org.apache.beam.sdk.transforms.Combine.CombineFn<InputT,AccumT,?> combineFn, org.apache.beam.sdk.coders.Coder<K> keyCoder, double sizeEstimatorSampleRate, boolean isGloballyWindowed)	Returns a grouping table that combines inputs into an accumulator with sampling SizeEstimators.
void	flush(org.apache.beam.sdk.fn.data.FnDataReceiver<org.apache.beam.sdk.values.WindowedValue<org.apache.beam.sdk.values.KV<K,AccumT>>> receiver)	Flushes all entries in this table to output.
long	getWeight()
void	put(org.apache.beam.sdk.values.WindowedValue<org.apache.beam.sdk.values.KV<K,InputT>> value, org.apache.beam.sdk.fn.data.FnDataReceiver<org.apache.beam.sdk.values.WindowedValue<org.apache.beam.sdk.values.KV<K,AccumT>>> receiver)	Adds the key and value to this table, possibly flushing some entries to output if the table is full.
PrecombineGroupingTable<K,InputT,AccumT>	shrink()	Returns a new object that is smaller than the object being evicted.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

combining

public static <K,InputT,AccumT> PrecombineGroupingTable<K,InputT,AccumT> combining(org.apache.beam.sdk.options.PipelineOptions options,
                                                                                                           Cache<java.lang.Object,java.lang.Object> cache,
                                                                                                           org.apache.beam.sdk.transforms.Combine.CombineFn<InputT,AccumT,?> combineFn,
                                                                                                           org.apache.beam.sdk.coders.Coder<K> keyCoder,
                                                                                                           boolean isGloballyWindowed)

Returns a grouping table that combines inputs into an accumulator. The grouping table uses the cache to defer flushing output until the cache evicts the table.

combiningAndSampling

public static <K,InputT,AccumT> PrecombineGroupingTable<K,InputT,AccumT> combiningAndSampling(org.apache.beam.sdk.options.PipelineOptions options,
                                                                                                                      Cache<java.lang.Object,java.lang.Object> cache,
                                                                                                                      org.apache.beam.sdk.transforms.Combine.CombineFn<InputT,AccumT,?> combineFn,
                                                                                                                      org.apache.beam.sdk.coders.Coder<K> keyCoder,
                                                                                                                      double sizeEstimatorSampleRate,
                                                                                                                      boolean isGloballyWindowed)

Returns a grouping table that combines inputs into an accumulator with sampling SizeEstimators. The grouping table uses the cache to defer flushing output until the cache evicts the table.

shrink

@Nullable
public PrecombineGroupingTable<K,InputT,AccumT> shrink()

Description copied from interface: Cache.Shrinkable

Returns a new object that is smaller than the object being evicted.

It is recommended to return an object that is at most half as large as the one being evicted. If null is returned then the object will be evicted.

Specified by:

shrink in interface Cache.Shrinkable<K>

getWeight

public long getWeight()

Specified by:

getWeight in interface org.apache.beam.sdk.util.Weighted

put

public void put(org.apache.beam.sdk.values.WindowedValue<org.apache.beam.sdk.values.KV<K,InputT>> value,
                org.apache.beam.sdk.fn.data.FnDataReceiver<org.apache.beam.sdk.values.WindowedValue<org.apache.beam.sdk.values.KV<K,AccumT>>> receiver)
         throws java.lang.Exception

Adds the key and value to this table, possibly flushing some entries to output if the table is full.

Throws:

java.lang.Exception

flush

public void flush(org.apache.beam.sdk.fn.data.FnDataReceiver<org.apache.beam.sdk.values.WindowedValue<org.apache.beam.sdk.values.KV<K,AccumT>>> receiver)
           throws java.lang.Exception

Flushes all entries in this table to output.

Throws:

java.lang.Exception