Package org.apache.beam.sdk.extensions.avro.io

Class DynamicAvroDestinations<UserT,DestinationT,OutputT>

java.lang.Object

org.apache.beam.sdk.io.FileBasedSink.DynamicDestinations<UserT,DestinationT,OutputT>

org.apache.beam.sdk.extensions.avro.io.DynamicAvroDestinations<UserT,DestinationT,OutputT>

All Implemented Interfaces:

java.io.Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData

public abstract class DynamicAvroDestinations<UserT,DestinationT,OutputT>
extends org.apache.beam.sdk.io.FileBasedSink.DynamicDestinations<UserT,DestinationT,OutputT>

A specialization of FileBasedSink.DynamicDestinations for AvroIO. In addition to dynamic file destinations, this allows specifying other AVRO properties (schema, metadata, codec, datum writer) per destination.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor	Description
DynamicAvroDestinations()

Method Summary

Modifier and Type	Method	Description
org.apache.avro.file.CodecFactory	getCodec(DestinationT destination)	Return an AVRO codec for a given destination.
@Nullable AvroSink.DatumWriterFactory<OutputT>	getDatumWriterFactory(DestinationT destinationT)	Return a AvroSink.DatumWriterFactory for a given destination.
java.util.Map<java.lang.String,java.lang.Object>	getMetadata(DestinationT destination)	Return AVRO file metadata for a given destination.
abstract org.apache.avro.Schema	getSchema(DestinationT destination)	Return an AVRO schema for a given destination.

Methods inherited from class org.apache.beam.sdk.io.FileBasedSink.DynamicDestinations

formatRecord, getDefaultDestination, getDestination, getDestinationCoder, getFilenamePolicy, getSideInputs, populateDisplayData, sideInput

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DynamicAvroDestinations

public DynamicAvroDestinations()

Method Detail

getSchema

public abstract org.apache.avro.Schema getSchema(DestinationT destination)

Return an AVRO schema for a given destination.

getMetadata

public java.util.Map<java.lang.String,java.lang.Object> getMetadata(DestinationT destination)

Return AVRO file metadata for a given destination.

getCodec

public org.apache.avro.file.CodecFactory getCodec(DestinationT destination)

Return an AVRO codec for a given destination.

getDatumWriterFactory

public @Nullable AvroSink.DatumWriterFactory<OutputT> getDatumWriterFactory(DestinationT destinationT)

Return a AvroSink.DatumWriterFactory for a given destination. If provided, it will be used to created DatumWriter instances as required.