Class AvroCoder<T>
- java.lang.Object
-
- org.apache.beam.sdk.coders.Coder<T>
-
- org.apache.beam.sdk.coders.CustomCoder<T>
-
- org.apache.beam.sdk.extensions.avro.coders.AvroCoder<T>
-
- Type Parameters:
T- the type of elements handled by this coder
- All Implemented Interfaces:
java.io.Serializable
- Direct Known Subclasses:
AvroGenericCoder
public class AvroCoder<T> extends org.apache.beam.sdk.coders.CustomCoder<T>ACoderusing Avro binary format.Each instance of
AvroCoder<T>encapsulates an Avro datum factory and schema for objects of typeT.The Avro datum factory and schema may be provided explicitly via
of(AvroDatumFactory, Schema)or omitted viaspecific(Class)orreflect(Class)in which case it will be inferred using Avro'sSpecificDataorReflectDataFor complete details about schema generation and how it can be controlled please see the
org.apache.avro.specificandorg.apache.avro.reflectpackages.To use, specify the
Codertype on a PCollection:PCollection<MyCustomElement> records = input.apply(...) .setCoder(AvroCoder.of(MyCustomElement.class));or annotate the element class using
@DefaultCoder.@DefaultCoder(AvroCoder.class) public class MyCustomElement { ... }The implementation attempts to determine if the Avro encoding of the given type will satisfy the criteria of
Coder.verifyDeterministic()by inspecting both the type and the Schema provided or generated by Avro. Only coders that are deterministic can be used inGroupByKeyoperations.- See Also:
- Serialized Form
-
-
Constructor Summary
Constructors Modifier Constructor Description protectedAvroCoder(java.lang.Class<T> type, org.apache.avro.Schema schema)protectedAvroCoder(java.lang.Class<T> type, org.apache.avro.Schema schema, boolean useReflectApi)protectedAvroCoder(AvroDatumFactory<T> datumFactory, org.apache.avro.Schema schema)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Deprecated Methods Modifier and Type Method Description Tdecode(java.io.InputStream inStream)voidencode(T value, java.io.OutputStream outStream)booleanequals(@Nullable java.lang.Object other)static AvroCoder<org.apache.avro.generic.GenericRecord>generic(org.apache.avro.Schema schema)Returns anAvroCoderinstance for the Avro schema.static org.apache.beam.sdk.coders.CoderProvidergetCoderProvider()Returns aCoderProviderwhich uses theAvroCoderif possible for all types.AvroDatumFactory<T>getDatumFactory()Returns the datum factory used for encoding/decoding.org.apache.avro.io.DatumReader<T>getDatumReader()Returns the DatumReader used for decoding.org.apache.avro.io.DatumWriter<T>getDatumWriter()Returns the DatumWriter used for encoding.org.apache.beam.sdk.values.TypeDescriptor<T>getEncodedTypeDescriptor()org.apache.avro.SchemagetSchema()Returns the schema used by this coder.java.lang.Class<T>getType()Returns the type this coder encodes/decodes.inthashCode()static <T> AvroCoder<T>of(java.lang.Class<T> clazz)Returns anAvroCoderinstance for the provided element class.static <T> AvroCoder<T>of(java.lang.Class<T> type, boolean useReflectApi)Returns anAvroCoderinstance for the given class, respecting whether to use Avro's Reflect* or Specific* suite for encoding and decoding.static <T> AvroCoder<T>of(java.lang.Class<T> type, org.apache.avro.Schema schema)Returns anAvroCoderinstance for the provided element type using the provided Avro schemastatic <T> AvroCoder<T>of(java.lang.Class<T> type, org.apache.avro.Schema schema, boolean useReflectApi)Returns anAvroCoderinstance for the provided element type using the provided Avro schema, respecting whether to use Avro's Reflect* or Specific* suite for encoding and decoding.static AvroGenericCoderof(org.apache.avro.Schema schema)Returns anAvroGenericCoderinstance for the Avro schema.static <T> AvroCoder<T>of(AvroDatumFactory<T> datumFactory, org.apache.avro.Schema schema)Returns anAvroCoderinstance for the providedAvroDatumFactoryusing the provided Avro schema.static <T> AvroCoder<T>of(org.apache.beam.sdk.values.TypeDescriptor<T> type)Returns anAvroCoderinstance for the provided element type.static <T> AvroCoder<T>of(org.apache.beam.sdk.values.TypeDescriptor<T> type, boolean useReflectApi)Returns anAvroCoderinstance for the provided element type, respecting whether to use Avro's Reflect* or Specific* suite for encoding and decoding.static <T> AvroCoder<T>reflect(java.lang.Class<T> type)Returns anAvroCoderinstance for the provided element type respecting Avro's Reflect* suite for encoding and decoding.static <T> AvroCoder<T>reflect(java.lang.Class<T> type, org.apache.avro.Schema schema)Returns anAvroCoderinstance for the provided element type respecting Avro's Reflect* suite for encoding and decoding.static <T> AvroCoder<T>reflect(org.apache.beam.sdk.values.TypeDescriptor<T> type)Returns anAvroCoderinstance for the provided element type respecting Avro's Reflect* suite for encoding and decoding.static <T> AvroCoder<T>specific(java.lang.Class<T> type)Returns anAvroCoderinstance for the provided element type respecting Avro's Specific* suite for encoding and decoding.static <T> AvroCoder<T>specific(java.lang.Class<T> type, org.apache.avro.Schema schema)Returns anAvroCoderinstance for the provided element type respecting Avro's Specific* suite for encoding and decoding.static <T> AvroCoder<T>specific(org.apache.beam.sdk.values.TypeDescriptor<T> type)Returns anAvroCoderinstance for the provided element type respecting Avro's Specific* suite for encoding and decoding.booleanuseReflectApi()Deprecated.kept for backward API compatibility only.voidverifyDeterministic()
-
-
-
Constructor Detail
-
AvroCoder
protected AvroCoder(java.lang.Class<T> type, org.apache.avro.Schema schema)
-
AvroCoder
protected AvroCoder(java.lang.Class<T> type, org.apache.avro.Schema schema, boolean useReflectApi)
-
AvroCoder
protected AvroCoder(AvroDatumFactory<T> datumFactory, org.apache.avro.Schema schema)
-
-
Method Detail
-
generic
public static AvroCoder<org.apache.avro.generic.GenericRecord> generic(org.apache.avro.Schema schema)
Returns anAvroCoderinstance for the Avro schema. The implicit type is GenericRecord.
-
specific
public static <T> AvroCoder<T> specific(org.apache.beam.sdk.values.TypeDescriptor<T> type)
Returns anAvroCoderinstance for the provided element type respecting Avro's Specific* suite for encoding and decoding.
-
specific
public static <T> AvroCoder<T> specific(java.lang.Class<T> type)
Returns anAvroCoderinstance for the provided element type respecting Avro's Specific* suite for encoding and decoding.
-
specific
public static <T> AvroCoder<T> specific(java.lang.Class<T> type, org.apache.avro.Schema schema)
Returns anAvroCoderinstance for the provided element type respecting Avro's Specific* suite for encoding and decoding.The schema must correspond to the type provided.
-
reflect
public static <T> AvroCoder<T> reflect(org.apache.beam.sdk.values.TypeDescriptor<T> type)
Returns anAvroCoderinstance for the provided element type respecting Avro's Reflect* suite for encoding and decoding.
-
reflect
public static <T> AvroCoder<T> reflect(java.lang.Class<T> type)
Returns anAvroCoderinstance for the provided element type respecting Avro's Reflect* suite for encoding and decoding.
-
reflect
public static <T> AvroCoder<T> reflect(java.lang.Class<T> type, org.apache.avro.Schema schema)
Returns anAvroCoderinstance for the provided element type respecting Avro's Reflect* suite for encoding and decoding.The schema must correspond to the type provided.
-
of
public static AvroGenericCoder of(org.apache.avro.Schema schema)
Returns anAvroGenericCoderinstance for the Avro schema. The implicit type is GenericRecord.
-
of
public static <T> AvroCoder<T> of(org.apache.beam.sdk.values.TypeDescriptor<T> type)
Returns anAvroCoderinstance for the provided element type.- Type Parameters:
T- the element type
-
of
public static <T> AvroCoder<T> of(org.apache.beam.sdk.values.TypeDescriptor<T> type, boolean useReflectApi)
Returns anAvroCoderinstance for the provided element type, respecting whether to use Avro's Reflect* or Specific* suite for encoding and decoding.- Type Parameters:
T- the element type
-
of
public static <T> AvroCoder<T> of(java.lang.Class<T> clazz)
Returns anAvroCoderinstance for the provided element class.- Type Parameters:
T- the element type
-
of
public static <T> AvroCoder<T> of(java.lang.Class<T> type, boolean useReflectApi)
Returns anAvroCoderinstance for the given class, respecting whether to use Avro's Reflect* or Specific* suite for encoding and decoding.- Type Parameters:
T- the element type
-
of
public static <T> AvroCoder<T> of(java.lang.Class<T> type, org.apache.avro.Schema schema)
Returns anAvroCoderinstance for the provided element type using the provided Avro schemaThe schema must correspond to the type provided.
- Type Parameters:
T- the element type
-
of
public static <T> AvroCoder<T> of(AvroDatumFactory<T> datumFactory, org.apache.avro.Schema schema)
Returns anAvroCoderinstance for the providedAvroDatumFactoryusing the provided Avro schema.The schema must correspond to the provided datumFactory's type.
- Type Parameters:
T- the element type
-
of
public static <T> AvroCoder<T> of(java.lang.Class<T> type, org.apache.avro.Schema schema, boolean useReflectApi)
Returns anAvroCoderinstance for the provided element type using the provided Avro schema, respecting whether to use Avro's Reflect* or Specific* suite for encoding and decoding.The schema must correspond to the type provided.
- Type Parameters:
T- the element type
-
getCoderProvider
public static org.apache.beam.sdk.coders.CoderProvider getCoderProvider()
Returns aCoderProviderwhich uses theAvroCoderif possible for all types.It is unsafe to register this as a
CoderProviderbecause Avro will reflectively accept dangerous types such asObject.This method is invoked reflectively from
DefaultCoder.
-
getType
public java.lang.Class<T> getType()
Returns the type this coder encodes/decodes.
-
getDatumFactory
public AvroDatumFactory<T> getDatumFactory()
Returns the datum factory used for encoding/decoding.
-
getDatumWriter
public org.apache.avro.io.DatumWriter<T> getDatumWriter()
Returns the DatumWriter used for encoding.
-
getDatumReader
public org.apache.avro.io.DatumReader<T> getDatumReader()
Returns the DatumReader used for decoding.
-
useReflectApi
@Deprecated public boolean useReflectApi()
Deprecated.kept for backward API compatibility only.- Returns:
- true if internal
AvroDatumFactoryis an instance ofAvroDatumFactory.ReflectDatumFactory
-
encode
public void encode(T value, java.io.OutputStream outStream) throws java.io.IOException
- Specified by:
encodein classorg.apache.beam.sdk.coders.Coder<T>- Throws:
java.io.IOException
-
decode
public T decode(java.io.InputStream inStream) throws java.io.IOException
- Specified by:
decodein classorg.apache.beam.sdk.coders.Coder<T>- Throws:
java.io.IOException
-
verifyDeterministic
public void verifyDeterministic() throws org.apache.beam.sdk.coders.Coder.NonDeterministicException- Overrides:
verifyDeterministicin classorg.apache.beam.sdk.coders.CustomCoder<T>- Throws:
org.apache.beam.sdk.coders.Coder.NonDeterministicException- when the type may not be deterministically encoded using the givenSchema, thedirectBinaryEncoder, and theReflectDatumWriterorGenericDatumWriter.
-
getSchema
public org.apache.avro.Schema getSchema()
Returns the schema used by this coder.
-
getEncodedTypeDescriptor
public org.apache.beam.sdk.values.TypeDescriptor<T> getEncodedTypeDescriptor()
- Overrides:
getEncodedTypeDescriptorin classorg.apache.beam.sdk.coders.Coder<T>
-
equals
public boolean equals(@Nullable java.lang.Object other)
- Overrides:
equalsin classjava.lang.Object- Returns:
trueif the twoAvroCoderinstances have the same class, type and schema.
-
hashCode
public int hashCode()
- Overrides:
hashCodein classjava.lang.Object
-
-